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ABSTRACT 


This  thesis  is  an  evaluation  of  the  Consolidated  Automated  Support  System  (CASS)  to 
determine  whether  it  is  likely  to  solve  a  host  of  long-standing  problems  with  use  of 
automatic  test  equipment  (ATE)  for  support  of  Naval  aircraft.  CASS  is  a  new  ATE 
program  nearing  the  end  of  full  scale  development  which  will  replace  all  current  ATE 
systems  and  many  smaller  manual  testers  to  form  a  single,  general  purpose  test  system  for 
all  intermediate  level  and  some  depot  level  avionic  testing.  It  is  intended  ultimately  to  be 
the  Navy  standard  test  system  for  all  NAVAIR  and  NAVSEA  requirements.  The  study 
involves  an  overview  of  the  environment  in  which  ATE  is  used,  a  history  of  ATE  problems 
and  development  leading  up  to  inception  of  the  CASS  program,  a  detailed  description  of 
the  program  itself,  a  description  of  the  repair  process  using  ATE,  and  an  evaluation  of  the 
CASS  program.  Its  conclusions  are  that  CASS  is  technically  capable  of  solving  or 
alleviating  virtually  every  significant  problem  affecting  use  of  existing  ATE  systems,  lead 
to  large  savings  in  weapon  system  support  costs,  and  improve  readiness  during  surges  in 
operating  tempo.  Its  success  is  very  vulnerable,  however,  to  decisions  made  during 
introduction  planning.  Such  decisions  indude  funding  cuts  which  cause  the  schedule  t 
o  slip  or  reduce  CASS  system  testing  performance  and  maintainability,  and  inaccurate  a 
nalysis  of  CASS  hardware  requirements  to  support  the  intended  testing  load  at  each  site. 
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I.  INTRODUCTION 


A.  PERSPECTIVE 

Computerized  Automatic  Test  Equipment  (ATE)  is  a  critical  element  of  logistic 
support  of  Naval  aircraft.  It  is  also  costly.  Over  $10  billion  must  be  spent  over  the  next 
20  years  to  replace  obsolete  ATE  systems  and  acquire  testing  capabilities  for  weapon 
systems  now  nearing  fleet  introduction.  [Ref.  l:p.  1.3] 

ATE  is  used  to  perform  intermediate  and  depot  level  electronic  testing  of  aviation 
electronic  components  and  circuit  boards.  It  has  been  increasingly  relied  upon  since  the 
first  major  ATE  system,  known  as  the  Versatile  Avionics  Shop  Tester  (VAST),  was 
introduced  to  the  fleet  in  1972.  Today,  intermediate  maintenance  shops  aboard  carriers 
operate  as  many  as  24  types  of  ATE  systems  plus  over  300  manual  electronic  testers  to 
support  die  diverse  testing  requirements  of  modem  weapon  systems.  [Ref.  l:p.  3-2] 

Unfortunately,  these  many  different  automatic  and  manual  testers  have  a  very  low 
level  of  equipment  standardization,  which  has  created  serious  supportability  problems. 
To  operate  and  maintain  them,  an  average  ship  must  carry  over  30,000  line  items  of  spare 
parts,  624  technical  manuals,  250  technicians,  and  dedicate  15,000  square  feet  of  space. 
[Ref.  2]  The  complexity  of  ATE  logistic  support  is  a  critical  concern  not  only 
because  of  its  high  cost,  but  because  ATE  supportability  is  closely  related  to  aircraft 
availability  and  stockage  requirements  for  avionic  system  spares. 

The  Consolidated  Automated  Support  System  (CASS)  is  a  new  ATE  system 
developed  to  replace  all  existing  ATE  systems  and  many  manual  testers  used  in 
intermediate  maintenance.  CASS  will  serve  as  the  Navy  standard  test  system  through  the 
early  twenty-first  century.  Nearing  the  end  of  full  scale  development  by  General  Electric, 
CASS  will  begin  operational  test  and  evaluation  in  mid- 1990,  with  initial  deliveries 
beginning  in  1992.  Fleet  introduction  should  begin  in  1994.  CASS  attempts  to  solve  the 
supportability  problems  of  existing  ATE  systems  through: 
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•  Improved  testing  throughput  capability  using  advanced  technology  and  testing 
concepts. 

•  Improved  reliability,  maintainability  and  standardization  of  hardware  and  software. 

•  Improved  ability  to  be  upgraded  and  reconfigured  to  satisfy  changing  testing 
requirements. 

•  Reduced  life  cycle  costs. 

•  Improved  support  system  stability  because  CASS  will  be  a  single  test  system 
serving  both  present  and  future  testing  needs.  This  will  enable  future  weapon 
systems  to  be  designed  for  testability  starting  at  program  initiation  and  reduce 
dependence  on  aircraft  contractors  to  provide  peculiar  support  equipment  and 
interim  support. 


B.  THESIS  OBJECTIVES  AND  RESEARCH  QUESTIONS 

The  purpose  of  this  thesis  is  to  evaluate  CASS  to  determine  whether  it  is  likely  to 
solve  the  host  of  long-standing  problems  with  use  of  automatic  test  equipment  for  support 
of  Naval  aircraft. 

The  primary  research  question  is: 

What  are  the  critical  problems  plaguing  the  current  family  of  testers  (FOT),  and  to 
what  extent  is  the  Naval  Air  Systems  Command  (NAVAIR)  solving  these  problems  in  the 
acquisition  and  fielding  of  the  Consolidated  Automated  Support  System  (CASS)? 
Relevant  subsidiary  research  questions  are: 

•  What  is  the  history  of  the  acquisition  of  ATE  in  Naval  Air?  What  were  the 
contrasting  weapon  system  support  philosophies  and  acquisition  strategies  for  the 
FOT  and  CASS? 

•  What  is  the  impact  of  ATE  supportability  deficiencies  on  weapon  system  support 
and  cost  of  spares? 


C.  METHODOLOGY 

Research  began  with  a  review  of  articles,  Navy  documents  and  data  relating  to  two 
areas.  First,  general  ATE  issues  were  examined,  including  evolution,  problems,  and  the 
relationships  between  ATE  supportability,  performance  and  weapon  system  spares 


requirements.  Second,  specific  CASS  program  issues  were  examined,  including  its  goals, 
acquisition  history  ,  attributes  and  introduction  plan. 

Visits  were  also  made  to  various  sites  to  attend  the  ATE  Cognizant  Field  Activity 
(CFA)  Review,  CASS  Program  Review  and  CASS  Logistic  Support  Analysis  (LSA) 
Review. 

Finally,  personal  and  telephone  interviews  were  conducted  with  representatives  of 
the  NAVAIR  CASS  Program  Office  and  introduction  team,  CASS  introduction  team 
members  at  the  Naval  Air  Engineering  Center  and  Naval  Aviation  Depot  Jacksonville, 
various  NAVAIR  and  type  commander  ATE  managers,  and  fleet  ATE  production 
personnel  at  NAS  Norfolk  and  aboard  USS  America. 


D.  THESIS  CHAPTER  SUMMARY 

Chapter  II  begins  with  an  introduction  to  the  role  and  environment  of  ATE  in 
avionics  repair.  It  then  traces  the  evolution  of  ATE  systems  and  problems  leading  up  to 
the  present  family  of  testers  and  to  CASS.  Chapter  EDI  describes  the  CASS  program  in 
detail,  including  its  goals,  current  status,  acquisition  history,  system  architecture,  and 
introduction  plan.  Chapter  IV  describes  the  intermediate  level  repair  process  using  ATE, 
to  lay  the  foundation  for  analyzing  the  performance  and  supportability  of  the  FOT  and 
CASS.  Chapter  V  compares  CASS  with  the  FOT.  Finally,  Chapter  VI  presents  a 
summary  of  the  research,  conclusions  and  recommendations. 
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II.  ATE  OVERVIEW 


This  chapter  is  an  introduction  to  the  environment,  function  and  evolution  of  ATE 
systems  in  support  of  Naval  aviation.  It  begins  with  a  description  of  automated  test 
equipment  within  the  context  of  the  avionics  maintenance  environment:  the  maintenance 
concept,  levels  of  repair,  ATE  facilities,  and  ATE  system  architecture.  The  chapter  then 
reviews  the  development  of  ATE  systems,  requirements,  problems  and  change  initiatives 
that  led  up  to  CASS. 


A.  ATE  SYSTEM  ROLE  AND  ENVIRONMENT 
1.  ATE  in  the  Avionics  Maintenance  Concept 

The  maintenance  concept  for  a  weapon  system  describes  its  overall  system 
support  environment.  It  includes  descriptions  of  maintenance  levels,  repair  policies, 
effectiveness  factors  and  logistic  support  requirements  such  as  test  equipment,  facilities, 
manpower  and  training,  and  supply  support.  Because  ATE  is  an  integral  part  of  the 
maintenance  concept  of  many  of  the  avionics  systems  presently  in  the  fleet  and  most  of 
the  systems  being  developed,  ATE  critically  influences  weapon  system  design  and  the 
other  logistic  elements  needed  to  support  it  throughout  its  life  cycle. 

Figure  1  illustrates  the  place  of  automatic  test  equipment  in  the  global 

classification  of  test  equipment.  [Ref.  3:p.  1.2]  As  this  taxonomy  shows, 

test  equipment  is  a  specific  catagory  of  support  equipment  that  is  formally  termed  test, 

measurement,  and  diagnostic  equipment  (TMDE).  It  includes  manual  electronic  test 

equpment  (ETE),  automatic  test  equipment  (ATE),  test  program  sets  (TPS)  and  calibration 

equipment  (reference  and  transfer  standards).  Each  of  these  four  catagories  of  test 

equipment  can  be  further  subdivided  as  shown  in  the  figure.  ATE  is  defined  as: 

Aviation  Support  Equipment  that  is  capable,  under  program  control,  of 
automatically  generating  and  independently  furnishing  programmed  stimuli, 
measuring  selected  parameters  of  an  electronic  or  electro-mechanical  item  being 
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PRIME  EQUIPMENT 
OPERATION  AND  SUPPORT 


Figure  1.  Taxonomy  for  Classifying  Test  Equipment. 
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tested  and  making  a  comparison  to  accept  or  reject  the  measured  values  in 
accordance  with  predetermined  limits... Included  in  this  definition  are  ATE  self-test 
and  self-check  test  program  sets  (TPSs)  and  weapon  system  or  equipment 
application  TPSs.  [Ref.  4] 

The  diversity  and  complexity  of  aircraft  weapon  systems  in  today’s  Navy  make 
automated  testing  an  essential  element  of  their  support,  especially  aboard  carriers. 
Intermediate  level  maintenance  shops  must  possess  a  broad  and  flexible  electronic  support 
capability;  shipboard  ATE  systems  typically  test  650  diverse  ’black  boxes’  and  2580 
subcomponents,  most  with  very  low  failure  rates.  [Ref.  l:p.  6-2]  To  do  so  manually 
would  be  a  monumental  task  in  terms  of  the  required  troubleshooting  time,  training, 
manpower  and  technical  publications.  Automation  has  demonstrated  its  ability  to  reduce 
test  times  by  as  much  as  30%  over  manual  testing  [Ref.  5:p.  63],  and  meet 
the  fundamental  constraints  posed  by  a  carrier’s  limited  space,  manpower,  and  off-ship 
logistics  pipeline.  For  these  reasons,  ATE  is  a  mainstay  of  the  Navy’s  organic 
maintenance  support  concept  for  avionics. 


2.  Level  of  Repair 

The  Naval  Aviation  Maintenance  Program  (NAMP)  is  a  six  volume  instruction 
that  sets  CNO  policy  for  aircraft  maintenance.  [Ref.  6]  It  defines  three  levels  of 
maintenance: 

•  Organizational:  Squadron  level  on-equipment  maintenance  functions  such  as 
aircraft  inspecting,  servicing,  trouble  shooting  and  replacing  parts  at  the  Weapons 
Replaceable  Assembly  (WRA)  level.  Avionics  WRA’s  are  the  ’black  boxes,’ 
packaged  functionally  and  for  ease  of  removal  and  replacement. 

•  Intermediate:  Support  organizations  that  perform  off-equipment  functions  such  as 
calibration,  limited  manufacturing,  and  repair  down  to  the  WRA  level  and  limited 
repair  of  avionics  Shop  Replaceable  Assemblies  (SRA)  such  as  circuit  boards.  ATE 
is  employed  extensively  at  this  level  for  fault  isolation  of  avionic  WRAs  and  SRAs. 
An  Aircraft  Intermediate  Maintenance  Department  (AIMD)  avionics  division  is 
typically  co-located  at  sea  or  ashore  with  the  squadrons  it  supports  and  consists  of 
200-400  Navy  technicians. 

•  Depot:  Support  organizations  that  perform  both  on  and  off-equipment  functions 
such  as  overhaul,  modifications,  calibration,  manufacturing,  repair  and 
condemnation  of  end  items,  including  aircraft,  WRAs  and  SRAs,  to  a  greater  depth 
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than  is  performed  at  the  O  or  I  level.  The  same  ATE  as  is  used  at  the  I  level 
performs  much  of  the  avionics  repair,  although  it  is  often  augmented  with  more 
capable  test  equipment.  They  are  not  co-located  with  squadrons/AIMDs,  have 
extensive  facilities,  and  typically  employ  several  thousand  civilian  technicians. 

NAMP  policy  dictates  that  maintenance  will  be  performed  at  the  level  "which 
will  ensure  optimum  economic  use  of  resources,  consistent  with  assigned  readiness  and 
availability  standards."  [Ref.  6:p.  8-7]  This  level  is  determined  by  Level  of  Repair 
Analysis  (LORA),  the  operational  need  for  quick  turn-around  of  aircraft,  and  the  physical, 
manning  and  training  constraints  of  O  and  I  level  facilities.  These  considerations  drive 
the  relative  capabilities  of  ATE  systems  developed  for  each  level.  Most  avionics  WRAs 
are  removed  and  replaced  at  the  O  level,  tested  and  repaired  at  the  I  level,  and  their  SRAs 
are  tested  and  repaired  at  either  the  I  or  D  level. 

Organizational  level  testing  capability  significantly  influences  I  and  D  level 
ATE  requirements.  Using  either  ’suitcase’  testers  or  Built  in  Test  Equipment  (BITE) 
designed  into  the  WRA,  extensive  fault  isolation  may  be  performed  by  the  flight  crew  or 
O  level  maintenance  technicians  without  removing  WRAs  from  the  aircraft.  This  may 
reduce  unnecessary  component  removals  and  aircraft  down  time.  Such  Built  in  Test 
(BIT)  capability  may  also  be  integrated  with  I  and  D  level  ATE  test  strategies.  If 
circuitry  for  self  test  is  contained  within  the  failed  component,  the  ATE  system  may 
monitor  it  to  speed  fault  isolation  and  to  reduce  hardware  and  software  requirements  of 
the  ATE.  Such  integration  is  achieved  in  newer  tester/weapon  system  mixes  such  as  the 
Intermediate-level  Avionics  Fault  Tree  Analyzer  (IAFTA)  for  the  F-18.  Through  such  a 
BIT/BITE  strategy,  the  Navy  seeks  to  reduce  and  ultimately  eliminate  I  level  ATE 
altogether  and  adopt  a  two-level  maintenance  concept  for  avionics.  Under  this  concept, 
O  level  technicians  will  isolate  faults  down  to  the  SRA  (circuit  card)  level,  which  will 
then  be  repaired  using  ATE  at  the  depot.  Weapon  system  components  will  be  packaged 
differently  than  they  are  today  to  facilitate  quick  removal  and  replacement  of  circuit  cards, 
perhaps  by  replacing  the  current  WRA  packaging  concept  with  banks  of  SRAs  accessible 
simply  by  removing  an  aircraft  panel.  This  ’O  to  D’  concept  will  be  more  technically 
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feasible  and  justified  (based  on  life  cycle  cost  analysis)  as  advances  continue  in  weapon 
system  reliability,  BIT  design,  manufacturing  and  packaging.  [Ref.  7] 

3.  Facilities 

ATE  must  perform  in  three  types  of  I  level  maintenance  facilities:  forward 
deployed  Mobile  Maintenance  Facilities  (8x8x20  foot  vans)  powered  by  portable 
generators;  permanent  shore  sites  (of  which  there  are  46,  though  many  of  the  smaller  ones 
currently  have  no  ATE);  and  ships.  Aircraft  carriers  are  by  far  the  largest  users  of 
forward  deployed  ATE,  although  there  is  a  growing  requirement  for  support  of  aircraft 
such  as  the  H-60  and  V-22  using  ATE  installed  in  vans  and  aboard  amphibious  ships. 

There  is  no  standard  facility  layout  or  standard  ATE  requirement  list  for  each 
type  of  AIMD  activity.  Each  shore  NAS  specializes  in  the  support  of  a  few  types  of 
aircraft  such  as  fighter,  attack,  ASW,  or  helicopter,  and  no  two  facilities  are  alike.  While 
aircraft  carriers  have  far  more  commonality,  they  nevertheless  vary  in  their  complement 
of  aircraft  because  of  differing  deckloads.  For  example,  many  older  ships  cannot  support 
F-14s,  so  several  combinations  of  F-18s  and  A-6s  have  been  used  to  meet  mission 
requirements.  The  result  is  that  every  sea  and  shore-based  I  level  ATE  suite  must  be 
tailored  to  the  systems  it  supports  and  undergoes  constant  change  as  aircraft  and  their 
weapon  systems  are  phased  in  and  out. 

The  carrier  ATE  shop  is  a  difficult  physical  environment  for  electronic  systems 
such  as  ATE  that  operate  at  high  power  and  generate  heat.  ATE  is  used  in  close  quarters 
and  faces  extremes  in  shock,  input  power  transients,  and  unreliable  air  conditioning 
systems.  ATE  must  perform  and  be  supportable  despite  the  inherent  stresses  of  this 
operating  environment. 

4.  ATE  Architecture 

Architecture  refers  to  the  packaging  of  the  required  set  of  functional 
components  of  the  ATE  system.  Functions  may  be  combined  in  one  package,  as  in  a 
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suitcase  tester,  or  they  may  be  modular  instruments  and  interface  devices,  as  in  large 
semi-permanently  mounted  ATE  systems. 

Every  ATE  system  shares  a  common  set  of  basic  functional  components.  It 
must  have  a: 

•  Control  processor:  A  computer  and  software; 

•  Operator  interface  device:  A  CRT  and  keyboard; 

•  Power  distribution  system; 

•  Suite  of  instrumentation  assets:  Devices  to  stimulate  and  measure  the  response  of 
the  unit  under  test  (UUT); 

•  Interconnection  Device  (ID):  The  electronic  and  mechanical  interface  between  the 
ATE  station  and  the  UUT. 

The  ID  is  often  a  large,  heavy,  electronic  device  set  on  a  cart.  It  contains 
active  circuitry  plus  interface  hookups  such  as  wiring  harnesses.  It  may  also  be  a  passive 
device  that  is  little  more  than  an  electrical  junction  box  and  cables. 

In  general  purpose  ATE  systems,  the  ID  is  one  functional  unit  in  the  complete 
Test  Program  Set  (TPS).  The  TPS  contains  everything  needed  to  adapt  a  particular  UUT 
to  an  ATE  station  and  perform  a  test,  including: 

•  Test  Program  (TP):  Application  software  containing  the  test  instructions  for 
component  stimulation  and  output  measurement.  It  may  be  stored  on  tape  (as  in 
VAST)  or  hard  disk  (single  or  stacked)  and  is  installed  or  called  up  to  run  the 
desired  test. 

•  Test  Program  Instruction  (TPI):  A  paper  or  software  description  of  the  operating 
procedures  to  connect  the  station,  UUT,  ID  and  other  devices  that  may  be  required, 
such  as  portable  power  supplies,  and  initiate  and  execute  the  test. 

•  Supplementary  Data:  Also  known  as  engineering  support  data  (ESD),  they  may 
include  test  strategies,  signal  interfaces,  functional  block  diagrams  and  diagnostic 
flow  charts  of  the  UUT  and  TPS  that  may  be  needed  to  complete  a  test. 

Figure  2  illustrates  these  TPS  components,  showing  a  CASS  station  in  the 
background.  [Ref.  8]  Because  every  WRA  and  SRA  tested  on  ATE  has  a  unique 
TPS,  some  5500  TPSs  currently  exist  to  support  NAVAIR  weapon  systems.  [Ref.  l:p. 
6-2] 
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Figure  2.  Test  Program  Set  Components. 
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B.  EVOLUTION  OF  ATE 


This  section  highlights  the  evolution  of  ATE  support  of  Naval  aircraft,  with 
particular  emphasis  on  the  problems  and  change  initiatives  that  led  to  development  of 
CASS.  These  problems  and  initiatives  are  not  merely  of  historical  interest,  but  are  also 
relevant  to  the  current  state  of  automatic  testing  because  every  generation  of  ATE  is  still 
in  the  fleet. 

1.  Peculiar  Support  Equipment 

Prior  to  the  introduction  of  common  (or  general  purpose)  ATE  systems  in  the 
early  1970s,  the  Navy  supported  its  avionic  testing  needs  using  manual  peculiar  support 
equipment  (PSE),  each  type  designed  for  testing  a  single  avionic  system.  The  rapid 
growth  in  diversity  and  complexity  of  weapon  systems,  however,  made  deployment  of 
system-specific  PSE  an  unworkable  concept  for  supporting  an  entire  aircraft  or  carrier  air 
wing,  for  the  reasons  discussed  below. 

PSE  testers  have  many  advantages  over  general  purpose  testers;  this  accounts 
for  their  continued  importance  in  our  support  equipment  inventory.  Acquisition  of  PSE 
rather  than  common  ATE  minimizes  technical,  schedule  and  cost  risks  because  the  avionic 
support  is  generally  developed  in  small  packages  to  support  each  sub-system  by  the 
weapon  system  contractor.  This  approach  optimizes  cost  effectiveness  at  the  subsystem 
level  because  PSE  can  be  developed  quickly  and  cheaply  and  can  be  tailored  to  provide 
the  exact  testing  capability  needed  with  no  frills. 

At  the  aircraft  or  carrier  air  wing  levels,  however,  use  of  PSE  developed  by 
the  weapon  system  contractor  is  a  case  study  in  sub-optimization.  It  results  in 
proliferation  of  different  types  of  test  equipment,  very  high  life  cycle  costs  for  the  weapon 
and  support  systems,  high  demands  for  manpower  and  facility  space,  short  test  equipment 
obsolescence  cycles,  redundant  development  costs,  little  competition  in  development  and 
production,  and  limited  flexibility  to  support  increased  testing  needs  during  a  surge  in 
operating  tempo.  Optimization  of  test  equipment  at  the  sub-system  level  comes  at  a  high 
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price  when  viewed  from  this  perspective,  making  acquisition  of  general  purpose  ATE  a 
more  cost  effective  alternative.  [Ref.  9] 

2.  VAST 

The  Versatile  Avionics  Shop  Tester  (VAST)  was  a  new  way  of  doing  business. 
It  was  the  first  general  puipose  ATE  aboard  carriers,  the  product  of  a  concept  adopted  by 
NAVA1R  in  the  mid-60s  with  the  goals  of  reducing  weapon  system  support  costs  and 
suppoitability  problems  inherent  in  reliance  on  the  growing  collection  of  PSE. 

The  first  production  contract  for  the  Versatile  Avionics  Shop  Tester  was 
awarded  in  1968,  and  VAST  stations  were  first  installed  aboard  carriers  and  at  shore  I 
level  and  D  level  sites  in  1972.  The  95  stations  bought  were  phased  in  between  1972  and 
1978  at  a  total  production  cost  of  $1  billion,  plus  at  least  another  billion  for  TPS.  [Ref. 
3:p.  5-11]  The  largest  ATE  ever  developed  for  DOD,  VAST  consists  of  10  bays  of 
hardware. 

VAST  was  designed  to  support  the  new  ’Grumman  Air  Wing’  of  F-14,  E-2  and 
S-3  aircraft,  including  over  150  WRAs  and  several  thousand  SRAs.  It  was  itself  initially 
supported  both  technically  and  logistically  by  the  contractors  (Harris  for  the  station  and 
Grumman  for  TPS).  VAST  was  expected  eventually  to  support  the  testing  and  calibration 
of  its  own  components.  Each  carrier  was  to  have  three  identical  stations,  with  as  many 
as  five  stations  going  to  some  large  shore  AIMDs.  [Ref.  10] 

Within  the  first  two  years  of  its  introduction,  however,  concern  arose  over 
VAST’s  ability  to  support  the  existing  and  emerging  testing  workload.  Because  VAST 
had  an  unexpectedly  low  throughput  and  required  TPSs  that  were  both  expensive  and  time 
consuming  to  develop,  many  test  requirements  initially  scheduled  for  VAST  support  were 
offloaded  to  manual  PSE  testers  that  were  already  developed.  New  general  purpose  ATE 
systems  were  also  developed,  so  that  by  1976  VAST  had  been  augmented  by  the 
Electronic  Module  Test  Console  (EMTC),  Digital  Module  Test  Console  (DMTC),  Central 
Air  Data  Computer  (CADC),  Automatic  Test  Set  (ATS),  Display  Automatic  Test  Set 
(DATS),  Hybrid  Automatic  Test  System  (HATS),  and  CAT  IID/CAT  HID  digital  SRA 
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testers.  [Ref.  5:p.  16]  A  Tailored  VAST’  was  also  created  by  reconfiguring  along 
functional  lines  to  test  only  digital  SRAs.  Even  with  all  these  additional  testers,  a  fourth 
VAST  was  soon  added  to  carriers  to  satisfy  workload  requirements. 
[Ref.  10:p.  4.5]  Figure  3  shows  this  complex  mix  of  testers  required  by  a  typical 
shipboard  AIMD.  [Ref.  5:p.  17] 

Despite  its  development  as  a  ’universal  tester’  for  the  S-3,  E-2  and  F-14,  by 
1976  VAST  actually  supported  87%  of  the  WRAs  and  27%  of  the  SRAs  for  these 
aircraft,  and  other  ATE  and  PSE  supported  the  remainder.  These  smaller  ATE  and  PSE 
systems  also  supported  the  remaining  carrier  deckload,  including  the  A-6E,  EA-6B,  KA- 
6D,  A-7B,  SH-3D  and  C-2A  aircraft.  [Ref.  5:p.  17] 

Part  of  VAST’s  difficulty  in  supporting  the  full  load  of  WRAs  and  SRAs  was 
that  the  VAST  station  itself  was  difficult  to  support.  The  system  was  complex  both 
technically  and  logistically,  and  was  supported  largely  by  the  contractor.  It  was  soon 
observed  that  whenever  the  VAST  stations  were  down  aboard  carriers,  within  a  few  days 
the  readiness  measures  of  all  three  weapon  systems  decreased. 

VAST  demonstrated  the  impracticality  of  using  a  single,  general-purpose  tester 
to  support  such  a  wide  spectrum  of  functions  and  large  testing  workload  given  the 
technology  of  the  time.  It  did,  however,  validate  the  concept  and  feasibility  of 
standardized  testing.  It  also  taught  a  major  lesson  about  support  of  support  equipment: 
there  is  a  direct,  dependent  relationship  between  ATE  availability  and  aircraft  readiness. 
[Ref.  11] 

3.  The  Marcy  Report 

In  response  to  widespread  problems  reported  by  the  fleet  with  VAST  and  other 
test  equipment,  the  Assistant  Secretary  of  the  Navy  for  Research  and  Development  (Mr. 
Marcy)  in  1975  requested  a  quick  study  of  problems  and  solutions.  An  ad  hoc  committee 
was  created,  composed  of  representatives  from  the  Naval  Material  Command  (NAVMAT), 
the  Systems  Commands,  their  field  activities,  the  fleet  and  industry  consultants.  [Ref.  5:p. 
2]  Their  1976  study,  known  as  the  Marcy  Report,  charted  a  new  course  in  the  way  the 
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Navy  acquired  and  managed  ATE  systems,  one  which  will  continue  to  guide 
decisionmakers  into  the  next  century. 

The  ad  hoc  committee  explored  the  entire  spectrum  of  problems  plaguing 
electronic  test  equipment,  including  hardware,  software,  TPS,  acquisition,  integrated 
logistic  support  (ILS)  management,  and  the  Navy  actions  required  to  solve  them.  They 
concluded  that  the  Navy  had  not  done  a  good  job  in  acquiring,  deploying  and  utilizing 
its  ATE  for  reasons  deeply  ingrained  in  the  way  the  Navy  did  business.  Chief  among 
these  was  the  lack  of  emphasis  on  ’front-end’  spending  during  the  initial  phases  of 
weapon  system  and  ATE  design.  Too  often  systems  were  developed  to  minimize  the 
initial  procurement  costs  even  when  higher  levels  of  spending  at  this  stage  (for  reliability 
improvements,  for  example)  would  result  in  lower  total  life  cycle  costs  and  fewer 
downstream  logistics  problems.  [Ref.  5:p.  68] 

Figure  4  summarizes  the  report’s  findings  in  a  problem/solution  matrix.  [Ref. 
5:p.  6]  These  20  problems  and  14  solutions  show  the  complex  and  often  overlapping 
technical  and  management  issues  the  ad  hoc  commitee  determined  must  be  addressed  to 
improve  ATE  operation  and  supportability  in  the  fleet.  The  report’s  recommendations 
included  the  following:  [Ref.  3:p.  5-12] 

•  Enforce  existing  Navy  policy  in  weapon  system  acquisition  to  ensure  their 
supportability. 

•  Educate  management  personnel  in  the  technical  and  management  issues  involved 
in  weapon  system  acquisition,  including  the  practical  problems  of  BIT  and  off-line 
ATE  hardware/software. 

•  Provide  quick  relief  to  the  fleet  by  (1)  initiating  engineering  changes  to  improve  the 
reliability  of  high-failure  rate  items  of  prime  equipment  as  well  as  ATE,  (2) 
establishing  "tiger  teams"  to  respond  to  ATE  problems,  (3)  developing  organic  test 
programming  capabilities,  and  (4)  prohibiting  deployment  of  ATE  without  prior 
approval  from  NAVMAT. 

•  Develop  a  new  family  of  general-purpose  ATE. 

•  Initiate  and  support  both  a  short-range  and  long-term  research  and  development 
program  in  automatic  testing  technology  under  supervision  of  a  central  NAVMAT 
ATE  management  office. 
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Figure  4.  Marcy  Report  Problem/Solution  Matrix. 


MANAGEMENT 


The  last  two  recommendations  formed  the  core  of  NAVAIR ’s  ATE 
development  plans  for  the  1980s  and  resulted  ultimately  in  the  CASS  program. 

4.  The  NAVAIR  ATE  Program  Plan 

NAVAIR  began  formulating  plans  for  the  1980s  and  beyond  by  assessing  the 
recommendations  of  the  Marcy  report,  two  subsequent  iTE  projects  involving  many 
DOD  and  industry  participants,  and  the  experiences  of  the  previous  decade  of  ATE 
support.  The  resulting  plans  were  documented  in  the  1979  NAVAIR  ATE  Program  Plan 
which  focused  on  achieving  seven  primary  goals:  [Ref.  3:p.  5-16] 

•  Integrate  ATE  program  management 

•  Improve  ATE  acquisition 

•  Design  avionics  for  testability  and  maintainability 

•  Minimize  the  variety  of  ATE 

•  Consolidate  and  improve  ATE  software 

•  Improve  the  quality  of  test  program  sets 

•  Attain  full  and  timely  organic  capability 

The  ATE  Program  Plan  a1c:o  lesponded  to  tasking  from  the  Office  of  the 
Secretary  of  Defense  to  develop  plans  to  standardize  weapon  and  support  systems, 
improve  weapon  system  support  and  readiness,  and  implement  sequential  system  upgrades 
through  Pre-Planned  Product  Improvement  (P3!)  efforts.  [Ref.  12:p.  7] 

NAVAIR  regarded  standardizing  ATE  aboard  carriers  as  the  central  objective 
of  this  plan,  because  standardization  encompasses  and  integrates  many  of  the  other 
program  goals.  To  achieve  this  objective,  the  plan  established  a  short  range  goal  and  a 
long  range  goal.  These  were,  first,  to  create  a  functional  family  of  common  ATE  to  serve 
through  the  1980s  and  next,  to  define  and  acquire  a  Consolidated  Support  System  (CSS) 
as  CASS  was  first  called,  to  serve  into  the  next  century.  [Ref.  l:p.  2-2] 


17 


5.  The  Short  Range  Goal:  A  Functional  Family  of  Testers 

To  achieve  an  immediate  reduction  in  proliferation,  NAVAIR  sought  to 
identify  a  group  of  existing  testers  most  able  to  accept  general  purpose  testing  functions 
to  support  a  broadened  range  of  weapon  system  types.  These  testers  would  form  a 
functional  family  of  testers  (FOT).  Program  managers  would  be  required  to  give 
preference  to  one  of  these  testers  in  planning  support  for  new  weapon  systems.  However, 
establishing  a  preferred  family  of  testers  was  slow.  It  took  until  1981  to  standardize  the 
ATE  selection  process  and  until  1982  to  publish  a  preferred  ATE  list.  [Ref.  3:p.  5-15] 
Many  of  the  24  general  purpose  ATE  systems  and  hundreds  of  PSE  types  now 
in  the  fleet  were  developed  after  the  decision  to  standardize  because  it  is  difficult  to 
enforce  exclusive  use  of  FOT  members  given  the  cost  and  schedule  pressures  of  weapon 
system  acquisition  and  the  advantages  of  procuring  PSE  support,  such  as  low  up-front 
costs  and  rapid  development  and  fielding. 

The  24  ATE  systems  are  classified  by  the  functions  of  the  weapon  systems 
they  support,  because  these  functions  dictate  the  test  requirements  and  instrumentation  of 
the  ATE.  Functional  classes  are  shown  in  Figure  5:  [Ref.  1] 


WRA  Support 

SRA  Support 

*  General  avionics 

*  Digital 

*  Inertial  navigation 

*  Analog  and  hybrid 

*  Electro-optical 

*  Electronic  warfare 

*  Radar/display 

Figure  5.  ATE  Functional  Classifications. 


The  current  preferred  family  of  testers  is  shown  in  Table  I. 

[Ref.  13:p.  3.3]  Also  shown  is  their  year  of  initial  delivery,  manufacturer, 

functional  classification,  and  original  weapon  system  application.  While  most  were 
initially  developed  to  test  a  single  weapon  system,  each  now  supports  a  broad  mix  of 
aircraft  types  and  many  different  WRAs  and  SR  As.  For  example,  the  CAT  DID  was 
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designed  as  a  digital  SRA  tester  for  the  F-14A,  but  now  also  supports  SR  A  testing  and 
some  WRA  testing  for  the  A-7E,  A-6E,  E-2C  and  EA-6B.  Others  were  developed  from 
the  start  as  standard  functional  testers,  such  as  the  Hybrid  Test  System  (HTS)  which  is 
designed  to  test  all  hybrid  and  analog  SRAs  not  testable  by  the  CAT  HID. 

Table  I.  The  Functional  Family  of  ATE. 


Year 

Name/Nomenclature 

Mfg. 

Functional  Type 

Initial 

Application 

1972 

VAST  (AN/USM-247) 

Harris 

Digital- Analog-Hybrid 

Common 

1975 

CAT  HID  (AN/USM-429(V)) 

Grumman 

Digital-Analog-Hybrid 

F-I4A 

1976 

1MUTS  H  (AN/USM-608(V)) 

Litton 

Inertial  Navigation 

F/A-18 

1979 

EOSTS  (AN/AAM-60(V-6)) 

Boeing 

Electro-Optics 

S-3A 

1980 

NEWTS  (AN/USM-458) 

Sanders 

Electronic  Warfare 

ALQ-126B 

1981 

RADCOM  (AN/USM-467) 

Grumman 

Radar 

E-2C 

1983 

ATS  (AN/USM-470(V-1)) 

Harris 

Digital- Analog-Hybrid 

F/A-18 

1983 

HTS  (AN/USM-484) 

Harris 

Hybrid 

Common 

1984 

ATS  (AN/USM-470(V-2)) 

Harris 

Digital-Analog-Hybrid 

SH-60 

1985 

AEWTS  (AN/USM-487) 

Honeywell 

Electronic  Warfare 

ASPJ 

6.  The  Long-Range  Goal:  CASS 

The  Consolidated  Automated  Support  System  is  the  outcome  of  the  long-range 
goal  established  by  NAVAIR  in  1979.  Research  and  development  work  actually  began 
after  the  1976  Marcy  Report  to  explore  new  testing  technology  in  recognition  of  two 
factors.  First,  the  Navy’s  existing  test  equipment,  including  test  systems  then  under 
development,  had  limited  capability  to  meet  the  advanced  testing  needs  of  avionics  to  be 
introduced  in  the  late  1990s,  so  new  systems  were  needed.  Second,  the  functional  family 
of  testers  would  be  able  to  provide  the  short  term  support  needed  until  a  new  general 
purpose  test  system  could  be  developed  and  fielded  in  an  orderly  and  controlled  manner. 
[Ref.  l:p.  2-2] 
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Advances  in  microelectronics  promised  to  be  both  a  challenge  and  an 
opportunity  in  this  long-range  development  effort,  presenting  many  problems  in  weapon 
system  measurement  and  diagnostics,  while  offering  many  opportunities  for  improved  test 
system  capabilities.  Research  and  development  focused  on  several  areas:  [Ref.  5:App. 
E] 

•  Avionic  design  for  testability,  including  BIT,  large  scale  integrated  (LSI)  circuits, 
self-test  and  self-repair  through  reconfiguration,  printed  circuit  boards,  non¬ 
electronic  equipment,  and  packaging. 

•  Advanced  ATE  concepts,  including  test  system  fault  isolation  (to  eliminate 
ambiguity  between  faults  in  the  ATE,  TPS  and  UUT),  improved  interface  devices, 
human  factors,  and  use  of  microcomputers  for  program  control  and  stimulation. 

•  ATE  software,  including  design  and  verification,  automation  of  documentation,  built 
in  checks  for  test  programs,  and  program  standardization. 

•  Automatic  test  program  generation,  including  development  of  circuit  analysis 
algorithms  and  failure  mode  analysis  techniques. 

•  Self-calibration  technology,  including  development  of  built-in  standards,  mobile 
calibrating  packs  and  automatic  calibration  techniques. 

•  Hardware  and  software  configuration  management,  including  development  of 
distributed  data  bases  and  mass  storage  to  manage  specifications,  engineering 
changes,  and  software  test  programs  and  documentation. 

•  ATE  technical  training,  including  development  of  interactive  programs  and 
simulators  to  improve  learning,  cut  costs  and  reduce  reliance  on  ATE  equipment  for 
operator  training. 

•  Microcomputer  testing  technology,  including  development  of  BIT  design  and 
acquisition  strategies  to  improve  testing  and  fault  detection/isolation  in 
microcomputer  based  electronic  subsystems. 

While  many  avionic  and  ATE  systems  acquired  in  recent  years  have  benefited 
from  these  R&D  efforts,  the  CASS  program  has  sought  to  integrate  the  state  of  the  art  in 
each  of  these  areas  to  achieve  an  optimal,  standardized  test  system.  The  outcome  of  this 
development  effort  is  the  subject  of  the  next  chapter. 
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m.  THE  CASS  PROGRAM 


This  Chapter  describes  the  CASS  program  in  terms  of  equipment,  acquisition 
strategy  and  introduction  plan  to  lay  the  groundwork  for  later  evaluation  of  the  system. 
It  is  organized  first  to  provide  program  goals  and  an  overview,  then  to  present 
progressively  greater  detail  about  its  relevant  attributes  and  acquisition/introduction 
strategy. 


A.  PROGRAM  OBJECTIVES 

The  Consolidated  Support  System  Project  was  established  in  1978  in  response  to 
the  NAVAIR  ATE  Program  Plan  to  provide  a  long-term  solution  to  the  many  historical 
ATE  problems  and  meet  the  challenge  of  emerging  testing  needs  through  the  year  2000. 

Its  objectives  are  two-fold;  first,  to  improve  throughput  (by  reducing  repair  cycle 
time)  which  will  result  in  improved  weapon  system  operational  availability  (A„),  readiness 
and  capability  to  meet  combat  sortie  requirements;  second,  to  standardize  ATE  hardware, 
software  and  support,  which  will  result  in  decreased  cost  of  ownership  of  both  avionic 
systems  and  avionic  test  equipment.  [Ref.  12:Addendum  A,  p.  1] 


B.  PROGRAM  OVERVIEW 

NAVAIR  has  called  CASS  and  the  T-45  jet  trainer  its  two  most  important 
acquisition  programs.  CASS  receives  this  high  priority  for  two  reasons.  First,  schedules 
for  new  weapon  systems  can  slip  with  far  less  damage  to  the  Naval  Air  mission  than  even 
a  small  slip  in  the  CASS  program  schedule.  The  latter  could  have  a  ripple  effect  on  the 
timing  of  many  weapon  system  introductions.  [Ref.  14]  Second,  failure  to 
achieve  its  technical,  logistic  or  performance  goals  could  seriously  impact  operational 
readiness  of  weapons  systems  targeted  for  CASS  support.  [Ref.  15:p.  8] 
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CASS  is  assigned  acquisition  category  (ACAT)  US  due  to  its  relatively  high  cost 
and  special  SECNAV  interest  (ACAT  I  is  the  highest  level,  generally  assigned  to  weapon 
system  programs  costing  over  $1  billion).  Right  on  schedule  so  far,  it  is  nearing  the  end 
of  Phase  II,  full  scale  engineering  development  (FSED),  by  General  Electric  Company  in 
Huntsville,  Alabama.  Its  next  major  milestone  (IHA)  is  set  for  August  1990,  when  low 
rate  initial  production  (LRIP)  of  81  CASS  stations  is  scheduled  to  begin.  Full  production 
will  result  in  a  total  inventory  of  775  CASS  stations  and  their  ancillary  hardware. 
Technical  evaluation  (TECHEVAL,  DT-IIC1)  and  pre -operational  evaluation  (OPEVAL, 
OT-IIA)  are  being  conducted  concurrently  during  Spring,  1990  at  the  Naval  Air  Test 
Center  in  Patuxent  River,  Maryland. 

The  remainder  of  this  overview  highlights  the  significant  events  in  the  development 
schedule  and  fleet  introduction  of  CASS. 

1.  Schedule 

Significant  future  events  are  summarized  in  the  program  schedule  shown  in 
Figure  6.  [Ref.  12]  Following  TECHEVAL,  CASS  stations  will  be  provided  to  the 
Operational  Test  and  Evaluation  Force  (OPTEVFOR)  for  full  OPEVAL  (OT-IIB).  In  this 
evaluation,  CASS  must  demonstrate  its  technical  performance  as  well  as  the  performance 
of  all  logistic  elements,  including  training,  spare  parts  and  technical  data  needed  to  keep 
CASS  operational.  TPS  developed  for  ’support  of  support’  (SOS)  for  CASS  SRAs  will 
be  also  be  tested.  (SOS  TPS  enable  one  CASS  station  to  test  failed  SRAs  from  another 
CASS  station.) 

After  successful  completion  of  testing  and  correction  of  any  problems,  a 
decision  will  be  sought  to  proceed  to  full  rate  production  (FRP)  in  mid  1991  (Milestone 
DIB).  The  Option  I  deliverables  shown  in  the  figure  are  the  CASS  stations  produced 
during  LRIP  (81  stations),  and  Option  II  and  HI  deliverables  are  those  produced  during 
FRP.  The  physical  configuration  audit  (PCA)  is  scheduled  to  take  place  in  early  1991, 
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which  is  the  earliest  date  the  Navy  will  accept  any  production  articles.  Initial  operating 
capability  (IOC)  will  occur  concuirently  with  the  material  support  date  (MSD)  and  Navy 
support  date  (NSD)  in  mid- 1992.  [Ref.  15: Attach.  1] 

2.  Fleet  Introduction  Overview 

Some  of  the  CASS  stations  produced  during  LRIP  are  scheduled  to  be 
delivered  beginning  in  late  1991  to  weapon  system  contractors  and  Naval  Aviation  Depots 
(NADEPS)  for  TPS  development,  and  to  Naval  Air  Maintenance  Training  Groups 
(NAMTRAGRUs)  for  operator  and  maintainer  training.  [Ref.  16:p.  1] 

Fleet  deliveries  to  sea  and  shore  AIMDs  for  operational  support  of  weapon 
systems  are  scheduled  to  begin  in  mid- 1993  after  all  logistic  elements  are  in  place.  These 
elements  include  training  of  fleet  technicians,  provisioning  and  delivery  of  spares,  and 
organic  depot  support.  There  will  be  no  phased  support;  all  logistics  will  be  delivered  up 
front  with  the  CASS  station,  and  all  elements  are  priced  into  the  single  firm  fixed  price 
development  contract. 

During  its  transition  into  operational  service,  CASS  must  accept  the  workload 
of  many  existing  testers  as  well  as  the  workload  of  new  weapon  systems  never  before 
supported,  such  as  the  A- 12  aircraft.  The  sequencing  of  TPS  development  and  stand-up 
of  CASS  at  each  site  will  be  based  on  the  timing  of  weapon  system  support  requirements 
and  the  urgency  of  replacing  obsolete  testers.  NAVAIR  has  drafted  an  introduction  plan 
covering  the  period  FY-90  to  FY-99  which  is  based  on  a  policy  requiring  all  new  aviation 
electronic  systems  to  be  supported  by  CASS  and  all  existing  systems  to  make  the 
transition  to  CASS  as  they  are  modified. 

The  Navy  Program  Decision  Meeting  which  granted  CASS  approval  to  enter 
full  scale  engineering  development  (FSED)  in  1986  also  directed  that  CASS  be  integrated 
into  the  surface  Navy  as  the  single  Navy-wide  standard  test  system,  with  NAVAIR  as  the 
lead  hardware  systems  command  for  equipment  acquisition.  While  its  current  operational 
requirement  is  for  support  of  avionic  systems  only,  it  will  be  rewritten  in  the  future  to 
reflect  this  expanded  tasking  and  CASS  development  will  continue. 
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C.  SYSTEM  DESCRIPTION 


The  CASS  system  is  a  modular,  reconfigurable,  computer  driven  automatic  test 
station  capable  of  providing  performance  verification  and  diagnostic  fault  isolation  for  all 
complex  electronic  components.  It  is  primarily  intended  for  I  level  shore,  carrier  and 
mobile  van  maintenance  environments,  but  will  also  be  used  at  the  depot.  Its  design  is 
intended  to  make  it  versatile  and  expandable  to  accept  the  Navy’s  testing  needs  through 
the  year  2011. 

1.  Configurations 

CASS  has  five  configurations  which  together  are  able  to  perform  the  same 
testing  functions  as  the  full  family  of  testers  (FOT).  These  are: 

•  Core  or  Hybrid  station  for  testing  general  digital,  RF,  and  analog  SRAs  and  WRAs. 

•  Electro-optical  (EO) 

•  Electronic  warfare  (EW) 

•  Digital/display  (DDS) 

•  Communications,  navigation  and  identification  (CNI) 

•  Radar 

There  is  also  a  portable  version,  the  CASS  Portable  Test  Set  (CPTS),  designed 
to  meet  Marine  Corps  forward  deployed  needs  and  future  NAVSEA  shipboard 
requirements.  The  portable  version  is  a  series  of  stacked  suitcases  that  may  be  configured 
to  be  functionally  identical  to  any  of  the  mainframe  CASS  stations  except  that  it  lacks 
some  power  protection  devices.  [Ref.  15] 

Figure  7  illustrates  these  configurations.  [Ref.  8]  The  hybrid  station  has  five 
racks  and  is  the  core  of  each  of  the  others.  It  can  be  reconfigured  into  any  of  the  other 
stations  by  adding  additional  circuit  cards  and  a  sixth  rack.  For  example,  the  hybrid  can 
be  converted  into  an  EO  station  by  adding  an  EO  rack  and  circuit  cards,  or  into  an  EW, 
CNI  or  radar  station  by  adding  a  radio  frequency  (RF)  rack  and  the  appropriate  additional 
circuit  cards.  The  CPTS  may  be  similarly  tailored  to  meet  the  functional  testing 
requirements  of  a  particular  forward-deployed  maintenance  activity. 
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Figure  7  also  shows  the  ancillary  equipment  required,  such  as  another  rack  to 
enable  pneumatic  component  testing,  a  printer,  and  an  EO  dolly  to  support  the  A-6E 
forward  looking  infra-red  (FLIR)/laser  turret.  The  next  section  explains  the  upper  portion 
of  the  figure  by  describing  the  station  architecture. 

2.  Architecture 

The  basic  station  functions,  such  as  stimulus,  measurement,  switching,  digital, 
power,  and  bus  interface  are  all  contained  in  the  hybrid  station,  with  additional  stimulus 
and  response  functions  provided  in  other  configurations.  Rather  than  using  a  separate 
instrument  for  each  of  these  functions  (such  as  a  digital  voltmeter)  as  in  many  current 
ATE  which  use  commercial  off-the-shelf  instruments,  CASS  uses  a  modular  approach, 
with  each  function  shown  in  Figure  7  performed  by  a  series  of  circuit  cards.  This  enables 
elimination  of  all  unneeded  functions  such  as  displays,  display  electronics,  packaging  and 
enclosures,  and  consolidation  of  common  functions  such  as  switching,  power  and  control 
electronics.  A  distributed  microprocessor  asset  controllers  combine  the  circuit  cards 
to  create  each  instrumcn  ^unction  needed.  The  single  Microvax  III  miniprocessor  serves 
as  the  station  controller  for  the  test  program,  peripherals  such  as  a  printer,  and  the  asset 
controllers,  which  are  linked  by  an  internal  ethemet.  An  external  ethemet  may  be  used 
to  establish  a  local  area  network  (LAN)  connecting  many  widely  distributed  CASS 
stations  and  a  management  information  system  such  as  NALCOMIS,  presently  being 
implemented  to  provide  I  level  maintenance  management  and  reporting.  [Ref.  8] 

The  five  core  racks  contain,  from  left  to  right:  [Ref.  17] 

•  Power  circuitry  needed  to  protect  the  station,  TPS  and  UUT  from  power  transients 
and  harmonic  distortion,  including  a  limited  battery  backup  and  circuitry  to  provide 
automatic  shutdown  of  the  station  for  longer  transients. 

•  Programmable  power  supplies  needed  for  testing  present  and  future  UUTs. 

•  Station  control,  containing  the  Microvax  HI  computer,  Winchester  drive  for 
magnetic  disks,  optical  disks  drive,  embedded  standards  for  self-calibration,  and 
operator  interface,  which  consists  of  a  plasma  display,  keyboard,  bar  code  reader 
and  trackball. 
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•  General  purpose  interface  (GPI)  for  adapting  to  TPS,  and  digital  test  unit  (DTU) 
and  DTU  power  supplies. 

•  Additional  DTU  power  supplies  and  space  for  growth  to  accommodate  new  testing 
requirements. 

These  hardware  components  are  designed  to  place  all  required  capability  in  the 
CASS  station,  eliminating  the  need  for  complex  active  circuitry  in  interconnection  devices 
(IDs),  as  is  often  required  for  existing  testers. 

There  is  a  high  degree  of  commonality  among  the  internal  components  of  the 
various  configurations,  as  illustrated  by  Figure  8.  For  example,  circuit  cards  (SRAs)  in 
the  HW  station  are  84%  common  to  the  hybrid.  To  convert  a  hybrid  station  into  an  EW 
requires  insertion  of  24  SRAs  internally  in  the  hybrid  station,  plus  connection  of  the  RF 
rack.  [Ref.  8] 

This  commonality  between  the  five  configurations  is  important  for  several 
reasons.  First,  it  enables  easy  reconfiguration,  as  discussed  above.  This  reconfigurability 
allows  the  suite  of  CASS  stations  at  a  site  to  be  tailored  to  meet  the  site-specific 
workload  of  WRAs  and  SRAs,  which  constantly  changes  with  the  aircraft  deckload  and 
introduction  of  new  avionic  systems.  Second,  commonality  is  the  key  to  standardizing 
ATE  logistic  support  because  it  results  in  fewer  types  of  components  and  fewer  spares, 
less  technical  data,  and  reduced  requirements  for  operator  and  maintainer  training.  Third, 
it  permits  limited  TPS  transportability  between  stations.  Transportability  means  that  a 
TPS  that  will  work  on  one  configuration  will  also  work  on  other  CASS  configurations. 
For  example,  some  TPSs  designed  for  the  CNI  configuration  will  also  work  on  the  radar 
configuration.  All  TPSs  designed  for  the  hybrid  will  transport  to  any  other  station.  This 
attribute  creates  considerable  flexibility  at  a  site;  workload  may  be  redistributed  to  other 
CASS  stations  when  a  station  goes  down,  and  the  site  has  improved  capacity  to  respond 
to  surges  in  operating  tempo  without  creation  of  long  testing  backlog.  [Ref.  8] 

3.  System  Software 

CASS  software  consists  of  the  following  elements,  described  below: 
[Ref.  18:  Annex  A] 
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Figure  8.  CASS  Configuration  Commonality. 


•  CASS  station  operating  system 

•  Weapon  system,  or  operational  TPS  programs  (OTPS) 

•  Self-maintenance 

•  Intermediate  Maintenance  Operations  Management  (IMOM)  System 

•  Operational  Management  System  (OMS) 

•  Support 

The  station  operating  software,  called  the  Test  Executive,  runs  on  the  Microvax 
m  computer.  It  accepts  code  from  the  TPS  program  to  execute  the  test,  controls  stimulus 
and  measurement  devices  to  exercise  the  UUT,  and  returns  measurement  data  to  the 
Microvax  in  station  controller  computer  for  evaluation.  It  also  controls  operator 
interfaces  such  as  the  plasma  display  screen  and  keyboard  and  accesses  a  resident  library 
on  a  mass  storage  device.  The  separate  microprocessor  asset  controllers  have  software 
to  control  individual  functional  circuit  cards  called  for  by  the  Test  Executive.  Programs 
are  written  in  a  variety  of  languages,  including  FORTRAN,  PASCAL,  and  assembly 
language. 

All  operational  TPS  software  used  by  a  particular  station  configuration  at  a  site 
are  stored  in  the  resident  library  and  may  be  drawn  down  on  call  by  the  operator  to  run 
a  test  on  a  UUT  or  may  be  transferred  to  other  stations  via  the  external  ethemet.  All 
OTPS  is  written  in  the  DOD  standard  test  language  known  as  ATLAS. 

CASS  uses  self-maintenance  software  to  perform  continuous  on-line  monitoring 
and  calibration  of  station  and  TPS  interface  circuits  simultaneously  with  UUT  testing. 
It  controls  calibration  standards  and  fault  detection  and  isolation  within  CASS,  plus  any 
line  loss  parametric  compensation  requirements.  The  self-tr  iintenance  software  monitors 
over  9,200  station  test  points  for  fault  detection  and  isolation.  [Ref.  8] 

The  Intermediate  Maintenance  Operations  Management  System  (IMOM) 
performs  automated  test  data  collection  and  storage  and  is  the  system  which  establishes 
the  communication  with  NALCOMIS  via  the  external  ethemet.  The  IMOM  system 
continuously  collects  data  reflecting  station  status,  operational  availability,  failed  station 
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hardware  that  was  isolated  using  the  self-maintenance  function,  station  usage,  operator 
identification,  and  the  results  of  UUT  testing.  [Ref.  19] 

The  Operational  Management  System  (OMS)  aids  I  level  maintenance 
personnel  with  management  of  production,  manpower  and  material  control.  This  includes 
assigning  UUTs  to  particular  workcenters  to  balance  workload  and  minimize  turn  around 
time  by  projecting/simulating  shop  performance  under  different  constraints,  woikloads, 
conditions  and  operating  policies.  OMS  software  contains  applications  utilities  such  as 
network  protocol  handlers,  input/output  functions,  and  mathematical  and  statistical 
packages.  Working  together,  the  IMOM  and  OMS  systems  are  designed  to  create  a 
completely  automated  ATE  shop.  [Ref.  2] 

CASS  support  software  is  used  to  assist  in  the  design,  development,  testing, 
and  support  of  station  software,  self-maintenance  software,  IMOM  and  OTPS.  Many  TPS 
development  software  tools  are  part  of  the  CASS  station  deliverables. 


D.  ACQUISITION  STRATEGY 

CASS  is  managed  by  NAVAIR  with  a  program  organization  (PMA-260)  rather  than 
by  the  support  equipment  acquisition  staff  (AIR -552)  as  has  been  done  for  all  previous 
ATE  acquisitions.  This  management  organization  was  directed  by  SECNAV  during  the 
pre-FSED  phase  in  March  1985,  when  CASS  was  designated  the  standard  ATE  for  the 
Navy.  [Ref.  12]  This  direction  reflects  the  facts  that  CASS’  scope  and  acquisition 
strategy  resemble  that  of  a  major  weapon  system  far  more  than  a  support  equipment 
system.  The  key  features  of  its  acquisition  strategy  are:  [Ref.  2] 

•  A  single  firm  fixed  price  (FFP)  contract  with  GE  for  FSED  and  three  years  of 
production  options  for  327  stations,  which  NAVAIR  may  change  up  or  down  by 
50%  to  meet  introduction  requirements  and  funding  constraints. 

•  A  contract  awarded  based  on  performance  specifications  and  equally  weighted 
technical,  acquisition/life  cycle  cost  and  logistic  support  factors. 

•  Military  standard  logistic  support  analysis  (LSA). 

•  A  dual  manufacturing  source,  Martin  Marietta  Technical  Services  Company 
(MMTSC),  to  ensure  head-to-head  competition  for  the  life  of  the  program. 
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•  Full  data  rights  and  validated  level  III  engineering  drawings. 

•  The  contractor  must  demonstrate  full  performance  (DT-IIC  and  OT-EIB)  prior  to 
acceptance  of  the  first  production  article. 

•  A  performance  warrantee  for  four  years  after  government  acceptance  of  each  end 
item. 

The  warrantee  clause  in  the  CASS  FSED  and  initial  production  contract  covers 
design  and  manufacturing  requirements,  defects  in  material  and  workmanship, 
technical/logistic  performance,  and  reliability  and  maintainability  (R&M).  R&M 
requirements  are  mean  time  between  failure  (MTBF)  of  148  hours,  mean  time  to  repair 
(MTTR)  of  2.0  hours,  100%  BIT  detection,  and  95%  BIT  isolation  to  one  SRA.  The 
OMS  system  will  be  used  to  collect  performance  data,  and  will  be  used  to  enforce 
technical,  logistic,  and  R&M  performance.  [Ref.  15:p.  6] 

1.  Acquisition  History 

The  initial  conceptual  phase  of  the  program  began  in  1978  with  small  studies 
designed  to  augment  the  Marcy  Report  by  clarifying  ATE  problems  in  procurement, 
management,  operation  and  future  test  requirements.  Acquisition  Phase  0  began  in 
August  1980,  when  CSS  was  approved  as  an  ACAT  HI  project  by  a  Navy  decision 
coordinating  paper  (NDCP).  On  31  December  1981,  five  contracts  were  competitively 
awarded  for  concept  exploration  and  definition.  Contractors  included  GE,  General 
Dynamics,  Sperry,  Grumman  and  Harris,  chosen  from  135  companies  who  responded  to 
the  request  for  proposal  (RFP).  Their  studies  were  completed  in  September  1983  and 
encompassed  system  requirements  analysis,  determination  of  appropriate  performance  and 
logistic  support  specifications,  systems  engineering  tradeoff  analysis,  and  creation  of 
computer  models  to  simulate  the  performance  of  their  proposed  systems. 

During  Phase  I,  Concept  Demonstration/Validation  (then  called  pre-FSED),  two 
industry  design  teams  competed  for  the  FSED  contract.  These  teams  were  headed  by 
Grumman  (with  GD  and  Hughes  as  partners)  and  GE  (with  Honeywell,  Northrup,  and 
Astrosystems  as  partners).  [Ref.  20]  GE  won  the  contract  in  December  1986. 
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Soon  after  that,  SECNAV  approved  Milestone  II  entry  into  FSED  in  a  February  1987 
Program  Decision  Memorandum  (PDM).  [Ref.  15:p.  1] 

2.  Full  Scale  Engineering  Development 

NAVAIR  awarded  GE  the  contract  for  FSED  based  on  equal  weighting  of 
technical,  logistics  and  cost  criteria  in  the  evaluation.  As  prime  contractor,  GE  awarded 
a  subcontract  to  Martin  Marietta  as  the  dual  manufacturing  source  (DMS)  in  December 
1988.  Under  this  contract,  GE  and  Martin  Marietta  will  share  production  of  the  first  327 
CASS  stations  with  a  60/40%  split.  As  an  incentive  to  ensure  full  DMS  qualification,  GE 
payments  are  contingent  on  Martin  Marietta’s  on-time  delivery  of  exact  copies  (verified 
through  PCA  and  operational  testing).  The  two  companies  will  then  go  into  head-to-head 
competition  for  the  remainder  of  the  life  of  the  system,  with  each  guaranteed  at  least  40% 
of  the  award.  Distribution  will  be  based  on  price  and  past  performance.  [Ref.  15:p.  4] 
Currently,  91  Martin  Marietta  employees  are  integrated  with  GE  employees  at  their 
Huntsville  site  for  training  in  co-assembly.  Martin  Marietta  is  constructing  a  32,000 
square  foot  production  facility  in  Americus,  Georgia.  [Ref.  16:p.  4] 

3.  Logistic  Support  Elements 

Because  most  of  the  problems  identified  by  the  fleet  in  the  Marcy  Report 
concern  supportability  issues,  logistic  supportability  has  been  a  major  CASS  program  goal 
since  its  inception.  Heavy  emphasis  was  placed  on  the  logistic  support  analysis  (LSA) 
process  to  ensure  that  support  continually  influenced  design  through  an  iterative  process. 
Additionally,  the  usual  concept  of  phased  support  during  system  introduction  was 
abandoned  during  negotiation  for  the  FSED  contract.  Instead,  all  logistic  support 
elements  were  priced  into  the  development  and  production  contract,  including 
development  of  training,  maintenance  publications  and  all  other  data  requirements, 
eliminating  the  need  for  interim  support  from  the  contractor. 

The  CASS  program  has  also  pursued  several  other  logistic  support  initiatives. 
CASS  was  developed  using  concepts  of  the  DOD  Computer-Aided  Acquisition  and 
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Logistic  Support  initiative  (CALS)  for  automated  digital  interchange  of  technical 
information  between  contractors  and  NAVAIR.  CALS  was  created  in  an  effort  to  reduce 
costs  and  improve  quality  and  integration  of  information  used  for  design,  manufacture  and 
1LS. 

The  most  noteworthy  product  of  this  effort  in  the  CASS  program  is  digitized 
technical  manuals  (paperless  publications).  Known  as  die  Automated  Technical 
Information  (ATI)  System,  all  of  the  many  volumes  of  technical  manuals  are  stored  on 
5.25  inch  erasable  optical  disks,  which  are  FSED  phase  deliverables.  Some  of  its  features 
include  menus,  help  files,  a  mouse/cursor,  icons,  multi-windowing,  browse  features, 
electronic  bookmarks,  and  automated  technical  publication  deficiency  reports  (TPDRs). 

There  are  many  advantages  to  this  paperless  publication  system,  including 
increased  operator  efficiency,  faster  updates,  and  reduced  storage  requirements  and 
transportation  costs.  It  also  allows  efficient  use  of  other  CALS  tools,  such  as  computer 
aided  design  and  the  LSA  process,  used  in  the  creation  and  quality  control  of  ATI 
publications.  [Ref.  16:p.  3] 

As  noted  above,  the  CASS  FSED  contract  requires  full  data  rights  and  delivery 
of  all  technical  data  (in  both  manual  and  digitized  formats)  including  level  HI  engineering 
drawings,  sufficient  for  future  competitive  reprocurement  of  end  items  and  spare  pans 
(with  the  exception  of  some  embedded  commercial  test  equipment  (CTE)).  All  data  and 
publications  must  be  delivered  after  they  are  validated  during  OPEVAL,  no  later  than  the 
physical  configuration  audit,  which  is  well  before  fleet  delivery.  [Ref.  12:p.  13] 

Formal  training  courses  will  also  be  evaluated  at  OPEVAL,  then  updated  to 
correct  problems  identified  prior  to  delivery  to  the  Naval  Training  Group 
(NAMTRAGRU).  Courses  and  personnel  classifications  have  been  created  for  CASS 
operator/maintainer  and  maintenance  technician.  An  embedded  training  module  is  also 
available  on  the  ATI  system  to  support  and  augment  the  formal  instruction  which  allows 
on-call  interactive  refresher  training  for  operators  and  maintainers.  A  CASS  innovation 
is  the  development  of  a  CASS  supervisor  course  which  includes  use  of  the  OMS, 
IMOMS,  ethemet  LAN,  ATI,  and  embedded  training  systems.  Manpower  planning  is  an 
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iterative  process  driven  by  the  introduction  plan  which  is  supported  by  the  Navy  Training 
Plan  (NTP)  scheduled  for  approval  in  August  1990.  [Ref.  2] 

The  CASS  program  is  also  the  prototype  for  a  new  approach  to  supply  support 
transitioning,  intended  to  allow  CASS  to  achieve  ASO  material  support  concurrent  with 
IOC  and  be  fully  supportable  before  it  hits  the  fleet.  This  ASO/NAVAIR  joint  venture 
aims  to  provision  spares  in  one  third  the  normal  time  by  paralleling  provisioning  events. 
Rather  than  following  a  sequential  process  of  ordering  end  items,  data  analysis  and  file 
loading,  and  finally  spares  procurement,  spares  will  be  delivered  by  GE  concurrently  with 
CASS  end  items.  This  will  achieve  economies  of  scale  as  well  as  eliminate  interim 
contractor  support.  The  danger  of  this  approach  is  that  the  first  spares  procurement  is 
right  in  the  middle  of  test  and  evaluation,  before  the  CASS  configuration  is  frozen.  As 
a  result,  GE  will  accept  the  risk  for  configuration  control  of  both  the  CASS  station  and 
spares  as  per  the  FSED  contract.  [Ref.  21] 

4.  Streamlining  and  Costs 

Acquisition  streamlining  initiatives  in  the  CASS  program  resulted  in  48% 
savings  in  the  FSED  contract  with  GE.  hi  the  approved  CASS  NDCP  of  18  April  1986 
NAVAIR  estimated  a  cost  of  $341  million  for  FSED  during  fiscal  years  87-91.  The  firm 
fixed  price  offer  from  GE  for  FSED  and  initial  production  was  $164.4  million,  which  is 
$176.8  million  less  than  the  estimate.  The  CASS  program  subsequently  received  a 
number  of  awards  in  recognition  of  this  cost  savings,  including  the  OSD  Acquisition 
Streamlining  Excellence  Award.  Streamlining  initiatives  were:  [Ref.  22] 

•  Converted  from  design  specifications  to  performance  specifications. 

•  Reduced  first  tier  specifications  from  126  to  95. 

•  Converted  all  3,000  second  tier  specifications  to  guidance  only. 

•  Reduced  the  contract  data  requirements  list  (CDRL)  from  359  to  258. 

•  Eliminated  the  engineering  development  model  (EDM). 

•  Eliminated  test  requirement  data  (TRD)  for  support  of  support  (SOS)  TPS. 
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•  Eliminated  interim  contractor  support  by  making  the  material  support  date  (MSD) 
and  Navy  support  date  (NSD)  concurrent  with  initial  operating  capability  (IOC). 

•  Required  the  contractor  to  propose  additional  CALS  initiatives. 

Among  the  most  important  features  of  the  CASS  acquisition  are  use  of  a  firm 
fixed  price  development  and  production  contract  containing  performance  rather  than 
design  specifications,  with  all  logistic  support  elements  priced  in.  They  were  the  key  to 
these  streamlining  achievements. 


E.  CASS  INTRODUCTION  PLAN 

The  CASS  Program  Office  (PMA-260)  and  the  Support  Equipment  Division  (AIR- 
552)  work  jointly  to  define  the  actions  and  milestones  necessary  to  successfully  introduce 
CASS  to  the  fleet.  This  effort  requires  identifying  existing  and  emerging  testing 
requirements  and  setting  priorities  to  establish  the  sequence  in  which  they  will  receive 
CASS  support.  From  this  plan,  the  transition  team  schedules  TPS  development  and  CASS 
station  delivery  to  each  site  in  the  right  quantities  and  configurations.  This  process  is 
iterative,  and  will  continue  throughout  the  CASS  life  cycle. 

The  following  paragraphs  examine  the  criteria  for  selecting  weapon  systems  to  be 
introduced  or  transitioned  to  CASS,  and  summarizes  the  first -round  CASS  introduction 
plan,  the  process  for  determining  site-specific  CASS  requirements,  and  the  TPS 
acquisition  strategy. 

1.  Weapon  System  Selection  Criteria 

Systems  to  be  transitioned  or  introduced  to  CASS  will  be  identified  based  on 
the  following  NAVAIR  policy,  established  in  1987  to  halt  development  of  any  new  ATE 
systems  and  to  establish  CASS  as  the  standardized  ATE  for  all  avionics  systems  of  the 
future:  [Ref.  23] 

•  All  electronic  systems  requiring  ATE  support  with  IOC  dates  during  FY-92  or 
beyond  will  be  supported  on  CASS. 

•  All  existing  electronic  systems  will  be  transitioned  to  CASS  as  they  are  modified 
or  upgraded. 
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•  All  electronic  systems  with  an  IOC  during  FY-90  through  FY-92  will  receive 
interim  support  until  supported  by  CASS.  (For  example,  the  F-14D  will  be 
supported  by  RADCOM  until  CASS  is  ready  to  accept  its  testing  requirements.) 

•  Testing  that  is  presently  done  on  obsolete  ATE  which  can  no  longer  be 
economically  supported  will  be  off-loaded  to  CASS  based  on  fleet  support  priorities 
and  economic  analysis. 

2.  Summary  of  Introduction  Plan 

Figure  9  is  the  product  of  this  policy.  It  identifies  the  weapon  systems 
scheduled  to  be  introduced  or  transitioned  to  CASS  support  through  fiscal  year  1999. 
[Ref.  24]  CASS  ’introduction’  applies  to  new  weapon  systems  never  before 
supported,  while  ’transition’  applies  to  existing  ATE  that  will  off-load  their  testing  of 
WRAs  and  SRAs  to  CASS  and  then  be  retired. 

Appendix  A  shows  the  schedule  for  these  introductions  and  transitions  (with 
the  weapon  and  ATE  systems  listed  in  alphabetical  order)  in  a  series  of  six  bar  charts. 
Many  of  the  weapons  systems  will  require  some  form  of  interim  support  if  they  reach 
IOC  before  CASS  and  CASS  TPS  are  available.  These  periods  are  shown  as  shaded  bars; 
the  beginning  of  the  full  clear  bars  indicate  the  start  of  full  CASS  support.  For  example, 
the  "F-14D  Misc"  WRAs  and  SRAs  will  be  supported  by  RADCOM  and  other  existing 
testers  from  the  first  quarter  of  1991  until  the  second  quarter  of  1995,  when  CASS  will 
begin  to  accept  some  of  the  testing  load.  Full  CASS  support  will  begin  in  the  second 
quarter  of  1998. 

The  AAM-60  electro-optical  station  will  be  the  first  to  be  transitioned 
(beginning  in  1994),  chosen  because  it  is  obsolete  and  difficult  to  support.  It  tests  WRAs 
for  the  A-6E  forward  looking  infra-red  (FLIR)  and  laser  system.  VAST  will  follow  later 
in  1994,  so  all  the  F-14A  and  S-3  WRAs  and  SRAs  it  supports  must  be  offloaded.  VAST 
has  a  high  priority  to  undergo  transition  because  it  must  be  removed  from  carriers  to 
make  room  for  CASS.  An  already  approved  Ship  Alteration  calls  for  replacing  one 
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Figure  9.  CASS  Candidate  Programs,  FY-90  to  FY-99. 
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VAST  with  one  CASS.  The  AAM-60  station  and  the  ASM-614  station  for  the  EA-6B 
will  also  be  one-for-one  replacements.  [Ref.  16:p.3]  A  total  of  19  CASS  stations  are 
expected  to  be  required  on  an  average  carrier  to  support  the  full  workload.  [Ref.  1  :p.  5-5] 

3.  Site  Planning 

The  goal  of  the  CASS  introduction  plan  is  to  optimize  the  CASS  testing 
resources  at  each  site.  To  do  this,  planners  must  select  the  right  weapon  system  UUTs 
to  be  introduced  or  transitioned,  and  then  deliver  tailored  CASS  configurations  to  handle 
the  workload  for  the  least  cost. 

The  process  of  optimizing  the  support  resources  at  each  site  will  be  managed 
using  three  computer  models  collectively  known  as  the  System  Synthesis  Model  (SSM). 
The  first  is  a  mapping  model  that  identifies  CASS  configurations  able  to  support  each 
UUT.  These  are  determined  by  matching  UUT  parametric  requirements,  such  as  power, 
signal  stimulus  and  measurement,  to  the  CASS  assets  that  provide  these  necessary 
functions. 

The  second  model  is  a  workload  planning  model  that  determines  CASS 
quantities  and  configurations  required  for  each  site.  The  key  input  parameters  are  CASS 
operational  availability,  the  configurations  specified  by  the  mapping  model,  and  UUT 
induction  rates.  Induction  rates  are  based  on  projected  aircraft  flying  hours  and  MTBF 
of  each  UUT.  This  model  is  discussed  in  greater  detail  in  Chapter  V.  [Ref.  24:p.  11] 

The  third  model  is  a  queuing  simulation  which  analyzes  the  sensitivity  of  the 
workload  model  output  to  changes  in  input  parameters.  The  simulation  will  be  used 
before  NAVAIR  makes  final  decisions  on  the  quantity  and  configuration  of  CASS  stations 
to  deliver  to  each  site.  [Ref.  25] 

These  three  models  are  vital  for  accurate  determination  of  site  requirements. 
The  dynamics  of  the  decision  process  are  very  complex  for  a  number  of  reasons.  First, 
thousands  of  UUTs  will  ultimately  be  supported  by  CASS.  Second,  TPS  transportability 
allows  many  UUTs  to  be  tested  on  multiple  configurations.  Third,  the  workload  at  each 
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site  will  undergo  constant  changes  as  transition  progresses  and  as  weapon  systems  come 
and  go,  forcing  stations  to  be  periodically  reconfigured. 

CASS  site  planning  must  also  deal  with  significant  constraints  which  may 
upset  plans  after  they  are  finalized,  such  as  the  availability  of  funds  for  CASS  production 
and  TPS  procurement,  the  production  output  of  GE  and  Martin  Marietta,  and  the 
performance  of  TPS  yet  to  be  developed.  These  constraints  may  affect  repair  throughput 
and  alter  projected  site  workloads.  [Ref.  24:p.  7] 

The  CASS  introduction  plan  is  particularly  vulnerable  to  slippages  in  TPS 
development.  For  example,  NAVAIR  calculated  the  impact  of  slipping  F-14D  TPS 
development,  which  is  the  top  CASS  requirements  driver.  (The  A- 12  program  is  the  next 
major  driver.)  Expenditure  of  $7.7  million  in  funding  budgeted  for  this  effort  in  FY-90 
is  at  risk  of  being  deferred  until  FY-91,  which  would  delay  F-14D  initial  organic 
capability  by  12  months.  This  slippage  would  increase  AVDLR  costs  by  approximately 
$30  million  to  sustain  interim  support  beyond  the  F-14D  MSD.  Because  it  would  also 
delay  the  offload  of  VAST  and  RADCOM,  this  slippage  would  disrupt  planning  for  the 
introduction  of  other  new  weapon  systems  because  CASS  cannot  be  installed  aboard 
space-constrained  carriers  until  these  other  ATE  stations  are  removed.  This  example  is 
intended  to  show  the  how  a  host  of  interdependent  factors  must  be  orchestrated  to  achieve 
an  orderly  CASS  introduction.  [Ref.  26] 

4.  TPS  Acquisition  Strategy 

The  key  elements  of  the  NAVAIR  (AIR-552)  TPS  acquisition  strategy  are: 

[Ref.  27] 

•  Development  contracts  will  be  firm  fixed  price,  sole  source  with  the  weapon  system 
prime  contractors. 

•  Production  contracts  will  be  openly  competed. 

•  The  government  will  provide  to  contractors  two  sets  of  UUTs  for  each  TPS,  CASS 
stations  and  TPS  development  tools  as  government  furnished  equipment  (GFE). 

•  The  government  will  provide  maintenance  and  spares  for  the  CASS  station  and 
UUT. 
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Thirty  months  is  the  targeted  lead  time  for  TPS  delivery,  from  identification 
of  a  requirement  using  the  process  described  above  until  the  date  it  is  needed  in  the  fleet.  (Ref.  28:p.  5] 
To  enhance  this  effort,  GE  has  delivered  a  suite  of  TPS  development  tools  as  part  of  the 
FSED  contract.  These  include  ATLAS  language  compilers,  translators,  editors, 
FORTRAN  debugging  programs,  automatic  test  program  generators  (ATPGs)  and  a  CASS 
Test  Executive  simulator.  GE  has  prepared  a  20  volume  TPS  developers’  guide  and  is 
preparing  a  TPS  development  training  course. 

The  standard  interface  environment  with  CASS,  combined  with  the  Test 
Executive  simulator  (which  runs  on  any  VAX  computer),  allows  early  design, 
development  and  testing  of  TPS  programs,  interconnection  devices  and  cabling  without 
a  CASS  station.  GE  demonstrated  this  capability  by  completing  most  of  the  A-6E 
FLIR/laser  TPS  development  and  fabrication  before  the  first  pre-production  station  was 
available.  A  similar  approach  is  intended  for  the  huge  development  effort  that  lies  ahead 
to  reduce  dependence  on  GFE  which  is  a  critical  cost  and  schedule  driver.  [Ref.  16:p. 
3] 
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IV.  THE  AVIONICS  REPAIR  PROCESS  INVOLVING  I  LEVEL  ATE 


Many  of  the  historical  ATE  problems  the  CASS  program  was  created  to  solve  are 
rooted  in  the  repair  process.  This  chapter  describes  the  supply/AIMD  repair  cycle  using 
existing  testers  to  clarify  the  functional  role  of  ATE  and  lay  a  foundation  for  the 
comparative  evaluation  of  CASS  in  the  next  chapter. 

A.  REPAIR  PROCESS  EFFECTS  ON  SPARES  REQUIREMENTS 

Intermediate  and  depot  level  repair  is  intended  to  fill  supply  shelves,  not  holes  in 
aircraft,  so  the  repair  process  is  intimately  related  to  weapon  system  spares  requirements. 
This  section  explains  that  relationship. 

The  I  level  maintenance  cycle  normally  begins  when  squadron  technicians  fault 
isolate  an  aircraft  system  failure  to  a  specific  WRA.  The  maintenance  concept  for  aircraft 
typically  requires  that  O  level  technicians  perform  all  on-equipment  work.  Any 
component  that  is  not  repairable  on  the  aircraft  is  removed  and  turned  in  to  the  local 
supply  department  and  a  replacement  is  requested.  This  requisition  is  ideally  filled  ’off 
the  shelf,’  while  the  failed  component  is  sent  to  the  local  AIMD  for  repair. 

The  AVCAL  (aboard  ship)  or  SHORECAL  (ashore)  includes  thousands  of  WRAs 
and  SRAs  stocked  to  meet  the  local  demand  for  spare  repairable  and  consumable  parts. 
The  fast  movers  receive  special  handling  and  are  kept  in  locations  close  to  their  squadron 
or  AIMD  customers  to  speed  their  delivery.  These  fast  movers  are  kept  in  the  rotatable 
pool  (for  WRAs),  module  pool  (for  SRAs)  or  pre-expended  bin  (for  consumables). 

The  full  AVCAL  is  a  level  of  safety  stock  designed  to  provide  90%  protection 
against  stockout  for  three  months  without  resupply.  Its  range  is  dependent  on  the  number 
of  types  of  aircraft  and  the  commonality  of  WRAs  among  aircraft  aboard  the  carrier,  and 
its  depth  is  dependent  on  expected  demand  (failure  rates)  during  the  three-month  pei  iod 
without  resupply.  In  the  case  of  I  level  repairables,  carrying  a  full  three-month  supply 
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can  be  avoided  by  repairing  WRAs  on  the  ship.  Thus,  the  90%  protection  level  for  these 
items  need  only  protect  for  the  repair  tum-around  time  (TAT)  for  the  I  level  activity. 
Additional  units  are  needed,  however,  to  accomodate  the  beyond  capabilty  of  maintenance 
(BCM)  rate.  Thus,  an  attrition  demand  level  is  also  provided  to  cover  the  three-month 
deployment  assumption. 

The  Aviation  Supply  Office  (ASO)  computes  the  size  and  composition  of  the 
AVCAL  using  "ASO  manual  rules."  The  average  AVCAL  quantity  of  each  repairable 
item  is  calculated  by  multiplying  its  average  repair  TAT  by  the  average  number  of 
removals  per  day.  Calculations  are  typically  based  on  Navy-wide  averages  for  removals 
per  flight  hour.  Removals  per  day  equals: 

(Removals/Flight  Hour)  *  (Flight  Hours/Aircraft/Day) 

*  (Number  of  Aircraft) 

This  average  is  used  in  the  Poisson  probability  distribution  to  determine  the  number 
of  units  of  an  item  which  are  needed  to  provide  90%  protection  over  TAT.  There  is  also 
a  special  consideration  for  low  demand  items  which  would  otherwise  have  zero  allowance 
under  these  rules. 

Figure  10  shows  AVCAL  levels  for  various  TATs  and  removal  rates  and  assuming 
no  BCMs  (thus  no  attrition)  calculated  by  the  Center  for  Naval  Analysis. 
[Ref.  29:p.  19]  It  illustrates  that  the  higher  the  removal  rate  for  a 

component,  the  greater  will  be  the  potential  for  decreases  in  TAT  to  cause  large  AVCAL 
reductions,  with  corresponding  percentage  reductions  in  weapon  system  life  cycle  costs. 
For  example,  given  a  component  with  0.7  removals  per  day  and  TAT  of  20  days, 
reducing  TAT  to  10  days  would  reduce  the  AVCAL  allowance  for  this  item  from  20  to 
11. 

The  BCM  rate  also  has  an  important  effect  on  sparing  requirements.  Table  B  shows 
the  average  quantities  of  a  component  in  the  off-ship  repair/resupply  pipeline  for  various 
values  of  BCM  rate,  assuming  one  removal  per  day,  a  five-day  TAT,  and  a  three-month 
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Turnaround  time  (days) 


Figure  10.  AVCAL  Levels  for  Various  TAT  and  Removal  Rates  (BCM  rate  =  0) 


support  period  without  resupply.  These  data  were  determined  by  the  Center  for  Naval 
Analysis.  [Ref.  30:p.  5]  The  table  shows  that  the  average  number  of  units 
needed  to  cover  attrition  drops  quickly  with  decreases  in  BCM  rate,  while  the  number  of 
units  needed  to  cover  the  repair  TAT  stays  fairly  constant  over  the  typical  range  of  BCM 
rates  between  0  and  0.3.  For  example,  reducing  BCM  rate  from  0.2  to  0.1  decreases  the 
number  of  units  in  the  resupply  pipeline  from  18  to  9  and  increases  units  in  the  repair 
pipeline  from  4.0  to  4.5. 

These  relationships  have  profound  implications  for  the  repair  process  and  ATE;  any 
improvements  in  equipment  or  process  which  reduce  TAT  or  BCM  rates  can  significantly 
reduce  the  quantity  and  cost  of  weapon  system  spares  required  to  achieve  the  same 
protection  against  stockout  during  a  deployment. 
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Table  II.  Average  Number  of  Units  in  Repair /Resupply  Pipelines 


BCM  Rate 

Repair 

Resupply 

Total 

.0 

5.0 

.0 

5.0 

.1 

4.5 

9.0 

13.5 

.2 

4.0 

18.0 

22.0 

.5 

2.5 

45.0 

47.5 

.8 

1.0 

72.0 

73.0 

.9 

.5 

81.0 

81.5 

1.0 

.0 

90.0 

90.0 

B.  THE  REPAIR  PROCESS:  INDUCTION  AND  SCREENING 

A  WRA  enters  the  AIMD  repair  cycle  as  a  component  awaiting  maintenance 
(AWM)  and  joins  the  repair  queue,  or  backlog.  The  aviation  material  screening  unit 
(AMSU)  of  the  AIMD  screens  the  WRA  by  part  number  to  determine  whether  the  AIMD 
has  repair  capability.  If  it  does  not,  the  WRA  is  returned  to  supply  for  shipment  to  depot 
maintenance  as  a  BCM  component.  If  full  or  partial  capability  exists,  the  WRA  is  sent 
to  the  appropriate  shop  for  induction  into  the  shop’s  repair  queue. 

Once  shop  technicians  receive  the  WRA  it  is  screened  again  and  is  either  inducted 
into  the  shop  repair  queue  or  returned  to  supply  (with  Production  Control  approval)  as  a 
BCM.  If  inducted  by  an  ATE  shop,  technicians  call  it  a  UUT  since  WRAs,  SRAs,  and 
ATE  station  repairables  all  intermingle  in  the  same  queue. 

At  the  same  time  that  a  component  is  received  by  the  shop,  data  (paper  or 
electronic)  about  the  induction  passes  through  Production  Control  (PC).  PC  assigns  a 
work  priority  for  running  WRAs  and  SRAs  on  ATE  based  on  two  rules:  criticality  of 
need,  followed  by  the  first  come,  first  served  rule.  In  assigning  a  status  based  on 
criticality  of  need,  expeditious  repairs  (EXREPS)  are  priority  1,  ’pool  critical'  and  ’pool 
zero  balance’  assets  are  priority  2,  and  all  others  are  priority  3.  [Ref.  6:p.  8-12] 
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EXREP  status  occurs  when  a  replacement  for  a  failed  repairable  part  that  downs  an 
aircraft  is  not  on  the  shelf  for  immediate  issue.  The  repair  pipeline  must  therefore  work 
to  fill  the  aircraft  ’hole,’  instead  of  the  supply  shelf.  Pool  critical  status  is  assigned  to  an 
inducted  component  when  its  stocking  level  drops  to  a  certain  percentage  of  the 
allowance,  and  pool  zero  balance  status  is  assigned  when  there  is  a  stockout.  Toward  the 
end  of  a  deployment,  this  should  occur  10%  of  the  time  with  the  standard  90%  protection 
level. 

An  additional  priority  status  exists  at  shore  AIMDs,  which  typically  support  carriers 
between  at  sea  periods  by  repairing  all  components  that  were  in  the  carrier’s  repair 
pipeline  at  the  time  the  ship  pulled  into  port.  As  the  carrier’s  deployment  date 
approaches,  these  ’repair  and  return’  inductions  compete  against  EXREPS  from  the  shore 
squadrons  for  priority  1 .  The  overall  effect  of  the  priority  system  is  to  create  a  lumpy 
demand  for  high  priority  repairs. 


C.  ATE  SETUP  AND  TESTING 

Components  inducted  to  the  ATE  shop  are  set  up  for  testing  by  connecting  the  UUT 
to  the  ATE  station  by  means  of  the  TPS.  Setup  operations  are  broken  down  into  external 
setup,  performed  off-line  while  other  UUTs  are  running  on  the  station,  and  internal  setup, 
performed  on-line  with  the  station  idle.  All  internal  operations  are  ’dead  time’  for  the 
station.  External  operations  include  collecting  the  necessary  TPS,  such  as  cables,  active 
power  supplies,  the  ID  and  test  program  disk,  and  setting  it  all  up  with  the  UUT  on  a 
roll-around  cart.  Intern -»l  operations  involve  plugging  it  all  in  to  the  station,  installing  the 
TPD,  and  downloading  the  program.  Station  self-test  and  calibration  are  also  internal 
operations  if  they  cannot  be  performed  while  the  station  is  running  another  program. 

The  setup  often  contributes  to  UUT  backlog,  because  many  operations  can  only  be 
performed  internally  when  the  station  is  idle.  As  each  site  typically  has  an  allowance  for 
only  one  TPS  and  one  ID  of  each  type,  the  individual  component  is  serviced  by  a  single 
channel  queuing  system  (one  server)  regardless  of  the  number  of  ATE  stations.  One  ID 
may  also  be  shared  between  multiple  TPS,  adapting  to  the  ATE  station  on  one  side  and 
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to  one  of  several  TPS  cable  sets  and  UUTs  on  the  other.  UUTs  which  share  a  common 
ID  cannot  be  run  concurrently  regardless  of  their  priority. 

Once  set  up,  the  test  program  is  downloaded  and  started.  Numerous  operator 
interventions  are  typically  required  during  the  course  of  the  test  to  answer  programmed 
questions,  manually  adjust  ID  power  supplies,  or  operate  switches  on  the  UUT. 


D.  FAULT  ISOLATION  AND  REPAIR 

TPS  software  by  design  does  not  provide  100%  fault  isolation  to  a  single  SRA. 
This  is  done  in  an  attempt  to  reduce  TPS  development  costs  and  program  run  time  (PRT). 
An  ATE/TPS  system’s  fault  isolation  capability  for  a  particular  UUT  is  instead  expressed 
as  the  probability  that  faults  can  be  isolated  to  an  ’ambiguity  group’  of  a  certain  size.  For 
example,  there  may  be  a  95%  probability  of  isolation  to  an  ambiguity  group  of  three 
SRAs. 

The  size  of  the  ambiguity  group  may  affect  the  quantity  of  SRAs  and  WRAs  needed 
in  the  AVCAL.  If  repair  is  performed  off-line  (after  disconnecting  the  UUT  from  the 
ATE),  all  SRAs  in  the  ambiguity  group  would  require  replacement,  even  though  it  is 
likely  that  only  one  has  a  fault.  If  repair  is  performed  on-line  (while  the  UUT  and  ATE 
are  still  connected),  the  ambiguity  group  affects  elapsed  maintenance  time  (EMT)  because 
SRAs  will  normally  be  replaced  sequentially;  if  three  SRAs  are  ’called  out,’  three  cycles 
must  be  made  through  the  program.  The  EMT  that  results  from  this  process  is  a  primary 
component  of  TAT  and  so  affects  WRA  sparing.  Ideally,  cost-benefit  analysis  should  be 
used  to  tradeoff  the  cost  of  TPS  fault  ambiguity  group  size  against  the  cost  of  AVDLR 
and  AVCAL  allowances.  If  such  a  tradeoff  were  made,  95%  isolation  to  three  SRAs  is 
expected  to  be  the  lower  limit  of  acceptable  TPS  performance.  [Ref.  31] 

Figure  1 1  is  a  flow  chart  for  the  testing,  fault  isolation  and  repair  process  in  an  I 
level  ATE  shop.  [Ref.  6:Chapters  8,  13  and  14,  and  Ref.  10:p.  C-3]  The  chart  shows  the 
steps  required  to  transform  components  inducted  for  repair  into  ready  for  issue  (RFI)  or 
beyond  capability  of  maintenance  (BCM)  WRAs  and  SRAs. 
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Figure  ll.Repair  Process  Flow  Using  I  Level  ATE 
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The  shortest  path  through  the  shop  is  the  case  of  a  ’false  alarm,’  which  occurs  when 
the  WRA  runs  ’end  to  end’  (from  the  beginning  to  the  end  of  the  test  program)  with  no 
fault  detected.  This  is  path  1  in  the  figure  and  leads  to  a  quick  RFI  if  there  are  no  other 
known  faults  and  the  WRA  is  not  a  ’Y  code’  (meaning  a  squadron  has  rejected  it  after 
a  previous  repair  attempt). 

Path  2  is  the  case  of  a  WRA  with  a  confirmed  failure  which  is  repaired  on-line. 
The  ATE  isolates  the  fault  and  SRAs  are  replaced  sequentially  with  multiple  cycles 
though  the  program  if  the  ambiguity  group  is  greater  than  one.  The  WRA  is  retested  end 
to  end  after  the  repair  is  complete  to  ensure  it  was  successful.  If  there  are  multiple 
failures,  the  retest  will  branch  into  another  fault  isolation  and  repair  cycle.  Thus  testing, 
fault  isolation  and  repair  operations  are  performed  iteratively  during  on-line  repair;  testing 
is  not  a  discrete  step  in  the  repair  cycle. 

Path  3  is  the  case  of  a  WRA  repaired  off-line.  Each  cycle  through  the  repair 
process  results  in  an  on-station  time  delay  for  disconnecting  and  reconnecting  the  WRA. 
The  size  of  the  ambiguity  group  does  not  affect  EMT  but  does  affect  the  number  of  SRAs 
which  must  later  be  tested  (some  of  which  are  false  alarms). 

Path  4  is  the  case  of  an  SRA  tested  on  ATE  but  completely  repaired  off-line.  When 
identified  by  the  ATE  as  failed  subcomponents  of  a  WRA,  they  make  a  loop  from  the 
ATE  shop  to  supply  and  are  re-inducted  to  AIMD  for  repair.  They  may  be  manually 
tested  by  a  specialized  component  repair  shop  or  enter  the  same  or  another  ATE  queue 
right  along  with  WRAs.  (For  example,  many  WRAs  tested  on  RADCOM  have  SRAs  that 
are  tested  on  the  CAT  HID  station.)  In  either  case,  the  tester  identifies  failed  circuit  card 
components  which  are  then  repaired  or  replaced  by  technicians  in  a  separate  micro¬ 
miniature  (2M)  repair  shop.  Just  as  WRA  repair  shops  are  not  intended  to  fill  holes  in 
aircraft,  SRA  repair  shops  work  to  fill  supply  shelves,  not  holes  in  the  next  higher 
assembly. 

WRAs  are  normally  repaired  on-line  because  off-line  repair  results  in  higher  SRA 
spares  requirements.  To  enable  sequential  replacement  of  SRAs  in  an  ambiguity  group 
during  on-line  repair,  special  SRAs  known  as  Maintenance  Assist  Modules  (MAMs)  are 
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kept  on  hand  in  the  ATE  shop.  MAMs  are  standard  SRAs  in  every  way,  but  are  procured 
for  high  failure  rate  SRAs  specifically  to  enable  fault  isolation  using  ATE.  After  the  test 
is  complete,  they  must  be  removed  and  replaced  with  standard  SRAs  obtained  from  the 
supply  department.  When  an  SRA  that  is  called  out  as  a  possible  failure  has  no  MAMs 
available  because  the  needed  MAMs  have  failed  or  are  not  stocked,  an  operator  typically 
cannibalizes  known  good  SRAs  from  awaiting  parts  (AWP)  WRAs  held  by  the  supply 
department  or  even  from  the  other  WRAs  in  the  AWM  queue.  The  operator  may  do  this 
with  permission  or  even  illegally  in  order  to  finish  the  test  before  breaking  down  the  TPS 
setup.  Cannibalized  SRAs  are  generally  not  removed  once  installed  in  contrast  to  MAMs, 
so  the  repair  step  is  completed  during  fault  isolation. 

The  frequency  of  the  program  entry  points  for  re-running  portions  of  the  test  and 
the  number  of  operator  interventions  it  requires  are  critical  determinants  of  the  time 
needed  to  complete  these  iterations.  Together,  they  determine  how  long  it  takes  to  return 
to  the  point  at  which  the  failure  was  detected  to  verify  its  correction  and  then  continue 
the  test.  If  the  design  of  the  ATE  allows  it,  experienced  operators  will  often  temporarily 
re-program  the  software  to  create  entry  points  closer  to  the  portion  that  must  be  re-run, 
reducing  the  time  for  each  iteration  by  tens  of  minutes. 

E,  ATE  AND  TPS  TROUBLESHOOTING 

Test  program  strategies  typically  assume  that  all  interconnecting  wires  in  the  TPS 
and  WRA  are  good  and  that  WRA  failure  modes  do  not  include  multiple  or  intermittent 
faults.  If  any  of  these  assumptions  are  wrong,  or  there  are  software  errors,  the  test 
program  may  generate  erroneous  SRA  failure  callouts.  The  operator  may  suspect  this  if 
a  UUT  will  not  pass  a  portion  of  the  test  after  multiple  replacements  of  an  SRA. 

This  happens  quite  frequently  with  certain  ATE/TPS  combinations.  For  example, 
NAS  Norfolk  AIMD  at  one  point  in  1988  documented  an  average  of  50  manhours  per  day 
troubleshooting  failures  in  the  many  hundreds  of  interface  pins  between  TPS  and  the 
RADCOM  and  CAT  HID  stations.  These  were  unreliable  and  very  vulnerable  to  damage 
from  power  fluctuations.  [Ref.  32] 
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The  operator  must  troubleshoot  the  entire  ATE/TPS/UUT  system  using  ATE  and 
TPS  off-line  self-test  capabilities  or  through  the  process  of  elimination.  Self-test 
capabilities  typically  consist  of  an  ATE  survey  self-test  which  checks  the  instilments  used 
for  that  test,  an  ID  confidence  check,  and  a  signature  identity  test,  used  to  ensure  all  the 
correct  equipment  is  installed.  The  success  of  these  checks  in  discriminating  between 
ATE,  TPS  and  UUT  faults  is  dependent  upon  their  comprehensiveness,  which  is 
frequently  not  sufficient  to  avoid  using  manual  troubleshooting  methods.  [Ref.  31] 

If  self-testing  does  not  isolate  the  fault,  ATE  station  failures  may  be  ruled  out 
through  the  process  of  elimination,  starting  by  breaking  down  the  whole  setup  and  moving 
the  TPS  and  UUT  to  another  of  the  same  type  of  station  if  there  is  one  at  the  site.  If  the 
station  is  ruled  out  as  a  failure,  TPS  failures  must  be  ruled  out  by  substituting  a  known 
good  UUT.  (Because  there  is  normally  only  one  TPS  of  each  type  per  site,  it  cannot  be 
substituted.)  Alternately,  the  operator/maintainer  may  turn  to  manual  troubleshooting 
using  probes,  ATE  and  TPS  hardware  schematics,  program  listings  and  test  diagrams  that 
lay  out  the  software  flow  contained  in  the  (often  unofficial  or  bootleg)  TPS  supplementary 
data  packages.  There  is  a  formal  training  track  for  this  troubleshooting  process  only  for 
VAST  technicians;  all  others  must  learn  on  the  job. 

If  the  long  process  of  troubleshooting  the  ATE/TPS/UUT  system  rules  out  station, 
TPS  and  SRA  problems,  the  shop  will  typically  declare  the  UUT  beyond  capability  of 
maintenance  (BCM). 

This  description  of  the  repair  process  is  important  for  understanding  the  factors 
which  influence  ATE  effectiveness.  The  next  chapter  is  the  evaluation  of  CASS.  It 
draws  on  this  and  each  of  the  foregoing  chapters  in  comparing  CASS  to  the  existing 
family  of  testers  to  determine  its  ability  to  solve  the  most  pressing  ATE  problems. 
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V.  COMPARATIVE  EVALUATION  OF  CASS 


A.  CRITERIA 

The  CASS  Acquisition  Plan  presents  the  operational  goals  for  the  program  which 
are  the  basis  for  evaluation  criteria  used  in  this  chapter.  They  are:  [Ref.  12:p.  Add.  A, 
P-  2] 

•  Achieve  a  100  percent  improvement  in  testing  throughput  capability  over  existing 
ATE. 

•  Provide  a  sustained  test  capability  with  higher  reliability  and  potential  for 
reconfiguration  to  satisfy  different  test  requirements. 

•  Reduce  proliferation  of  ATE  and  improve  logistics  supportability  and 
maintainability  through  standardized  hardware  and  software. 

•  Reduce  ATE  and  avionic  system  life  cycle  costs. 

These  goals  are  interrelated.  For  example,  reliability,  maintainability,  and  logistic 
supportability  are  key  determinants  of  throughput.  However,  the  cost  of  ATE  stations 
must  be  traded  off  against  the  cost  of  avionic  system  spares  in  the  AVCAL/SHORECAL. 
Many  of  these  relationships  were  illustrated  in  describing  the  repair  process  in  the  last 
chapter  and  are  further  clarified  in  this  chapter. 

CASS  is  evaluated  against  the  following  three  criteria: 

•  Turnaround  time  (the  reciprocal  of  throughput); 

•  Expandability  to  accept  emerging  testing  requirements; 

•  ATE  system  life  cycle  costs. 

These  criteria  encompass  all  four  of  the  above  goals.  The  remainder  of  this  analysis 
evaluates  expected  CASS  performance  against  the  criteria  in  comparison  with  the  family 
of  testers. 
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B.  TURNAROUND  TIME 

To  achieve  the  CASS  goal  of  100%  improvement  in  throughput,  turnaround  time 
(TAT)  must  be  reduced  by  50%.  Evaluation  of  the  ability  of  CASS  to  achieve  this  goal 
requires  analysis  of  the  ATE  attributes  that  influence  it,  and  comparison  of  CASS  to 
existing  testers  with  respect  to  these  attributes. 

TAT  of  weapon  system  components  through  the  local  I  level  repair  process  is  the 
total  time  from  removal  of  a  suspected  failed  WRA  from  an  aircraft,  or  SRA  from  a 
WRA,  until  the  WRA/SRA  is  RFI.  When  the  repair  process  employs  ATE  systems,  TAT 
is  built  up  from  several  components: 

•  Program  Run  Time  (PRT):  The  time  required  for  a  no  defect  end-to-end  test, 
including  programmed  operator  interventions. 

•  Elapsed  Maintenance  Time  (EMT):  The  total  time  that  a  UUT  occupies  a  station. 
It  includes  PRT,  troubleshooting  time  (while  swapping  MAMS  or  isolating 
station/TPS  failures),  time  spent  looking  for  parts  to  use  in  troubleshooting,  and 
repair  time  (while  replacing  SRAs  or  making  adjustments). 

•  Logistics  Delay  Time  (LDT):  Time  spent  in  the  ATE  backlog  waiting  for  station 
availability,  in  a  supply  awaiting  parts  locker,  and  in  the  supply  transportation  and 
administrative  processing  pipeline. 

The  key  ATE  system  attributes  affecting  these  times  include  the  test  strategy  and 
computer  capabilities  (which  affect  PRT  and  EMT),  logistic  supportability  (which  affects 
EMT  and  LDT),  and  the  number  of  ATE  stations  (which  affects  LDT).  The  following 
sub-sections  compare  CASS  and  members  of  the  FOT  in  these  attributes. 

1.  Test  Strategy 

CASS  TPS  development  plans  call  for  use  of  functional  test  strategies  rather 
than  the  parametric  test  strategies  now  used  in  most  TPS  (although  CASS  is  capable  of 
both).  [Ref.  33]  Parametric  testing  checks  all  input  and  output  signals  from  a 
UUT  sequentially  to  determine  if  they  comply  with  specifications.  It  is  a  strict  test  that 
requires  the  UUT  be  restored  to  essentially  factory  condition  to  be  declared  RFI, 
regardless  of  whether  a  detected  deviation  from  specification  caused  the  failure  reported 
by  the  squadron.  Functional  testing,  on  the  other  hand,  screens  the  component  only  for 
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failures  that  inhibit  its  proper  operational  performance.  It  is  in  many  ways  a  ’confidence 
check’  or  quick  verify  routine  that  checks  most-likely-to-fail  functions  first,  and  then  tests 
in  progressively  greater  depth  to  fault  isolate  to  an  ambiguity  group.  [Ref.  30:p.  1] 

A  functional  test  strategy  is  faster  than  parametric  in  both  PRT  and  time 
required  for  fault  isolation  and  results  in  lower  BCM  rates  because  it  does  not  require  the 
component  to  meet  factory  specifications.  It  may  gain  further  speed  and  program 
simplification  by  monitoring  the  BIT  circuits  within  a  WRA  for  fault  detection  and 
isolation.  It  may,  however,  cause  shorter  mean  time  between  removal  (MTBR)  of  WRAs 
because  the  test  may  fail  to  detect  true  faults  that  would  be  detected  by  parametric  testing, 
and  result  in  issue  of  WRAs  that  will  not  work  well  in  the  aircraft.  Functional  testing 
will  reduce  weapon  system  spares  allowances  in  the  AVCAL/SHORECAL  to  the  extent 
that  it  reduces  components  in  the  repair  cycle  (through  shorter  TAT)  and  resupply 
pipeline  (through  lower  BCM  rates)  without  an  offsetting  increase  in  spares  due  to  their 
reduced  reliability.  [Ref.  30:p.  1] 

The  Intermediate-Level  Avionics  Fault  Tree  Analyzer  (IAFTA)  ATE  system 
used  to  support  the  F- 18  aircraft  employs  functional  test  strategies  analogous  to  what  are 
planned  for  CASS  TPS  development.  Comparison  of  IAFTA  to  the  Avionics  Test  Set 
(ATS)  it  replaced,  which  uses  a  traditional  parametric  strategy,  shows  the  potential  impact 
on  TAT  of  replacing  other  parametric  testers  with  CASS. 

A  study  conducted  by  the  Center  for  Naval  Analysis  compared  the  TAT,  BCM 
rates  and  MTBR  of  14  F-18  WRAs  using  IAFTA  and  ATS.  Data  was  collected  primarily 
from  two  shore  sites  which  had  one  of  each  tester.  These  were  used  almost  equally  to 
repair  the  14  WRAs.  Their  results  are  summarized  in  Table  HI.  Total  TAT  using  IAFTA 
was  less  than  half  the  TAT  using  ATS  (averaging  6.5  days  compared  to  2.5  days),  as 
shown  in  the  second  column.  BCM  rates  using  IAFTA  dropped  to  a  fraction  of  those 
achieved  on  ATS  (averaging  10%  compared  to  less  than  1%),  as  shown  in  the  third 
column.  Some  components  tested  on  IAFTA  showed  slightly  reduced  reliability,  indicated 
by  the  increased  WRA  removal  rates  shown  in  the  forth  column. 
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Table  III.  Comparison  of  the  Effects  of  Functional  Versus  Parametric  Test  Strategies 
(IAFTA  vs  ATS). 


WRA 

Nomen. 

TAT 

ATS/IAFTA 

BCM  % 

ATS/IAFTA 

Removal 

Rate 

ATS/IAFTA 

#Sp 

ATS/L 

ares 

*FTA 

ADC 

6.5 

3.5 

2.5 

0.0 

.089 

.307 

2 

2 

RPYC 

2.9 

3.2 

1.2 

.158 

.477 

m 

6 

DataLink 

2.0 

2.3 

0.0 

0.0 

.113 

.062 

i 

0 

SDC 

6.3 

mm 

| 

m 

.269 

.737 

3 

5 

SDR 

8.2 

5.5 

8.6 

0.0 

.551 

.630 

8 

5 

CSC 

6.1 

2.8 

4.8 

0.0 

.328 

.481 

3 

v.  3 

HI 

7.0 

3.1 

30.5 

1.2 

.551 

.690 

15 

3 

DDI 

7.0 

17.3 

0.4 

.568 

.572 

10 

3'  ' 

HUD 

5.5 

3.6 

12.1 

5.0 

.458 

.259 

6 

2 

SMP 

6.6 

2.1 

8.0 

0.0 

.193 

.113 

1 

Gun  e/d 

6.2 

mm 

6.5 

0.0 

.169 

2.23 

8 

Wing  e/d 

8.2 

1.8 

20.8 

0.0 

.243 

.132 

6 

HI 

Fus  e/d 

4.9 

2.8 

11.8 

0.0 

.240 

.136 

sa 

2 

Pyln  e/d 

6.9 

2.3 

13.7 

0.0 

.193 

.410 

4 

2 

Total# 

78 

43 

Spares 

Tot  Cost 

$6.2 

3.1 

($  Mil) 

The  analysis  used  the  ASO  sparing  model  discussed  in  the  last  chapter  to 
calculate  the  change  in  AVCAL  spares  that  resulted  from  the  combination  of  these  three 
factors.  The  fifth  column  shows  AVCAL  spares  quantities  by  WRA  using  IAFTA  and 
ATS,  and  totals  the  quantities  and  spares  costs  at  the  bottom.  To  support  14  F-18  aircraft 
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aboard  a  carrier,  cost  of  spares  for  the  14  WRAs  tested  on  IAFTA  dropped  by  50%,  from 
$6.2  million  to  $3.1  million. 


In  analyzing  these  results  to  make  predictions  about  the  performance  CASS 
will  achieve,  several  points  must  be  considered.  First,  the  study  controlled  for  variables 
such  as  LDT,  which  may  substantially  affect  TAT,  but  did  not  break  out  PRT  or  EMT 
as  separate  measures.  As  a  result  it  is  difficult  to  specify  all  the  causes  of  the  50% 
reduction  in  TAT.  Differences  in  reliability  and  maintainability  of  IAFTA  and  ATS  may 
have  contributed  to  the  change  (although  the  BCM  rate  reduction  can  be  tied  directly  to 
the  functional  test  strategy  since  no  other  variables  are  likely  to  affect  it).  Second,  it  is 
not  clear  that  these  results  can  be  extrapolated  to  predict  results  for  the  whole  range  of 
WRAs  and  SRAs  that  CASS  will  test.  These  may  have  BIT  capabilities  and  levels  of 
complexity  that  vary  substantially  from  the  14  F-18  WRAs. 

Despite  these  questions  which  cloud  the  analysis,  this  study  leads  to  the 
conclusion  that  using  a  functional  test  strategy  with  CASS  is  likely  to  reduce  TAT,  BCM 
rate  and  cost  of  weapon  system  spares  for  at  least  some  complex  UUTs  or  those  with 
extensive  BIT  circuits. 

2.  Computer  Capability 

Improvements  in  computer  capabilities  between  earlier  generations  of  ATE  and 
CASS  may  significantly  reduce  the  time  required  for  several  testing  and  management 
functions.  These  capabilities  include  significantly  expanded  computer  processing  power, 
which  allows  both  faster  testing  and  simultaneous  performance  of  several  operator 
controlled  tasks,  and  management  tools  never  before  available,  such  as  the  OMS  and 
IMOMS  systems. 

The  influence  of  computer  processing  power  on  testing  time  is  shown  through 
comparison  of  VAST  and  RADCOM.  While  the  shift  to  a  software  orientation  in  test 
programs  between  VAST  development  in  the  late  1960s  and  RADCOM  in  the  late  1970s 
greatly  improved  the  ease  with  which  programs  can  be  updated,  PRT  increased  for  many 
UUTs.  For  example,  the  PRT  for  the  E-2C  main  display  unit  (MDU)  averaged  20 
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minutes  on  VAST,  but  45  minutes  on  RADCOM.  One  reason  for  this  is  that  RADCOM 
digital  analysis  is  performed  by  the  station  computer,  which  consequently  becomes  easily 
bogged  down  and  cannot  perform  any  other  task  at  the  same  time.  VAST,  by  contrast, 
performs  most  digital  analysis  within  its  digital  sub-system,  freeing  its  station  computer 
for  overall  test  control.  The  resulting  difference  in  computer  speed  may  become 
significant  in  terms  of  EMT  when  a  complete  test  or  portion  of  a  test  must  be  repeated 
numerous  times  during  on-line  WRA  repair  (using  the  process  illustrated  by  the  flow 
chart  in  the  last  chapter).  [Ref.  32] 

CASS,  like  VAST,  frees  its  station  computer  for  overall  test  control  by 
incorporating  the  series  of  microprocessor  sub-systems  known  as  asset  controllers.  CASS 
also  benefits  from  advances  during  the  1980s  in  computer  and  communications 
technology,  including  faster  computer  chips,  increased  memory  and  the  internal  and 
external  ethemet.  These  improvements  give  CASS  processing  power  which  will  allow 
many  simultaneous  computer  controlled  functions. 

The  CASS  operator  directs  these  functions  through  a  ’multi-tasking  windows 
interface’  rather  than  the  ’single-tasking  test  based  interface’  used  in  earlier  systems. 
[Ref.  31]  A  multi-tasking  interface  may  increase  station  utilization  by  moving  many  UUT 
setup  and  teardown  functions  from  internal  (performed  with  the  station  idle)  to  external, 
which  may  keep  the  station  engaged  in  actual  testing  for  more  of  its  available  operating 
time. 

This  multi-tasking  capability  may  be  illustrated  by  considering  a  CASS  station 
with  three  UUTs  to  be  run.  The  operator  sets  up  the  first  UUT  and  starts  its  testing,  then 
enters  data  for  the  next  UUT.  While  performing  the  first  test,  the  station  prepares  for  the 
second  by  calling  down  the  TPS  software  and  maintenance  publications  from  its  own 
memory  or  that  of  another  CASS  station  in  the  local  area  network  (LAN).  As  soon  as 
the  first  test  is  complete,  it  is  physically  disconnected  and  the  second  UUT  is  connected 
and  the  test  is  started.  The  IMOM  system  automatically  collects  and  stores  test  data  on 
the  first  UUT,  which  the  operator  can  then  enter  on  the  paper  maintenance  action  form, 
type  it  into  the  NALCOMIS  terminal,  or  uplink  it  in  electronic  form  to  NALCOM1S. 
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While  performing  these  tasks,  the  station  can  prepare  for  testing  the  third  UUT.  When 
there  are  more  than  three  UUTs  to  be  run,  the  station  and  operator  may  engage  in  a 
continuous  process  of  simultaneous  pre-test,  test  and  post-test  operations.  [Ref.  19] 

While  performing  the  testing  and  data  management  functions  for  the  three 
UUTs,  the  computer  may  also  be  used  for  embedded  training  of  technicians,  self¬ 
calibration  using  built-in  standards,  and  continuous,  automatic  self  testing/diagnostics. 
Together,  these  automated  systems  are  likely  to  reduce  EMT  by  increasing  operator 
efficiency  and  capability  during  testing. 

Finally,  the  IMOMS  and  OMS  systems  together  have  the  capability  to  make 
AIMDs  of  the  future  fully  automated  avionic  repair  facilities.  If  the  OMS  network  of 
CASS  stations  is  linked  to  a  minicomputer  in  production  control  (either  using 
NALCOMIS  or  a  specialized  ATE  network  computer  able  to  provide  software  storage  and 
a  communication  node),  the  ATE  shop  supervisor  or  production  chief  may  determine  in 
real  time  or  by  a  report:  [Ref.  19] 

•  The  availability  rate  and  fault/calibration  status  of  every  station  and  TPS. 

•  CASS  station  utilization  rates,  for  use  in  distributing  workload. 

•  The  status  of  any  in-work  UUT. 

•  Details  of  repair  actions,  PRT  and  EMT  of  any  UUT  or  type  of  UUT  from  a  library 
of  test  logs  within  the  OMS  data  base  management  system. 

•  Which  stations  are  most  efficient  at  running  a  particular  type  of  UUT. 

These  management  tools  may  substantially  increase  the  efficiency  and 
flexibility  of  CASS  production  managers  in  responding  to  increased  operating  tempo  or 
to  unexpectedly  large  volumes  of  UUTs  inducted  into  some  shops  (while  leaving  others 
idle).  In  contrast  to  existing  systems  which  have  little  such  surge  flexibility  and 
consequently  often  have  large  backlogs  and  long  TATs,  the  transportability  of  TPSs 
across  a  CASS  network  managed  using  the  EMOMS  and  OMS  systems  allows  workload 
to  be  evenly  distributed  and  effectively  controlled,  minimizing  logistic  and  administrative 
delays. 
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3.  Supportability 

Supportability  is  the  degree  to  which  an  ATE  system  can  be  supported,  in 
terms  of  characteristics  of  the  prime  test  equipment,  such  as  reliability  and 
maintainability,  and  the  effectiveness  of  the  elements  of  its  logistic  support. 
[Ref.  34:p.  16]  Because  the  majority  of  the  problems  identified  in  the 
Marcy  Report  concern  supportability  issues,  supportability  became  a  cornerstone  of  CASS. 
The  first  Navy  ATE  system  to  fully  benefit  from  the  Logistic  Support  Analysis  (LSA) 
process,  CASS  was  developed  with  heavy  emphasis  on  both  the  iterative  process  of 
designing  for  supportability  from  the  earliest  stages  of  the  program,  and  on  delivery  of 
total  organic  support  by  the  IOC  date.  This  section  will  evaluate  the  likely  results  of  that 
emphasis  in  terms  of  reducing  TAT. 

ATE  and  TPS  supportability  affect  repair  TAT  in  two  ways.  First,  the 
reliability  and  self-test  capability  designed  into  the  ATE  system  affects  EMT  because,  to 
complete  a  test,  operators  must  fault  isolate  not  only  the  UUT  but  any  station  and  TPS 
failures  that  occur.  Second,  ATE  reliability,  maintainability  and  logistic  support  affect 
ATE  system  availability,  which  influences  UUT  backlog  and  the  LDT  portion  of  TAT. 
The  following  paragraphs  analyze  these  two  effects. 

a.  Support -ihility  Effects  on  EMT 

CASS  may  significantly  reduce  EMT  in  comparison  to  existing  testers 
because  of  its  continuous  on-line  self  test  capability.  Members  of  the  FOT  have  only  off¬ 
line  self  test  programs  used  during  start-up  or  when  a  problem  is  suspected,  and  generally 
have  limited  TPS  fault  detection  capability.  The  significance  of  this  difference  is  clarified 
by  recalling  the  description  of  current  ATE/TPS  troubleshooting  practices  presented  in  the 
last  chapter. 

When  replacing  an  SRA  does  not  fix  a  UUT  failure  identified  by  a  test 
program,  the  operator  must  determine  whether  the  problem  is  in  the  UUT  (perhaps  in  the 
wiring  rather  than  an  SRA),  the  TPS  or  the  ATE.  Often  this  requires  extensive  manual 
troubleshooting  which  is  very  dependent  upon  availability  of  ATE  and  UlJTs  for  sub- 


stitution,  operator  skills  and  the  adequacy  of  maintenance  manuals.  Increased  reliability 
of  ATE  and  TPS  will  reduce  the  frequency  of  this  testing  interruption  while  self-testing 
capability  improvements  may  reduce  their  length. 

An  example  of  the  impact  of  supportability  factors  on  EMT  is  illustrated 
by  Table  IV,  which  shows  that  EMT  increased  for  several  high  failure  rate  E-2C 
components  when  they  were  offloaded  from  VAST  to  RADCOM  at  NAS  Norfolk  AIMD. 
[Ref.  32]  For  example,  EMT  for  the  HARS-SDC  (a  signal  data  converter)  tripled, 
increasing  from  8.1  hours  on  VAST  to  24.4  hours  on  RADCOM  (three  full  shifts  of 
station  time). 

Table  IV.  Comparison  of  VAST  and  RADCOM  EMT. 


WRA 

VAST 

RADCOM 

Change 

Nomenclature 

EMT  Hrs 

EMT  Hrs 

Factor 

MDU 

5.4 

7.5 

1.4 

HARS-SDC 

8.1 

24.4 

3.0 

UMD 

2.1 

11.4 

5.4 

ADC 

5.5 

9.8 

1.8 

Differences  in  test  programs  cannot  account  for  the  large  increases  in 
EMT  because  RADCOM  TPSs  are  virtually  identical  to  those  for  VAST.  Nor  are 
differences  due  to  computer  speed  because,  although  PRT  increased  between  VAST  and 
RADCOM  for  some  UUTs,  these  increases  are  measured  in  minutes  while  EMT  increases 
are  measured  in  hours.  For  example,  the  PRT  of  the  E-2C  main  display  unit  (MDU) 
shown  in  Table  III  increased  from  20  to  45  minutes  when  it  transitioned  from  VAST  to 
RADCOM  support.  (PRT  did  not  increase  by  a  consistent  percentage  for  all  UUTs, 
however.)  [Ref.  32]  Such  PRT  increases  may  have  a  multiplied  effect  on  EMT  when 
all  or  part  of  the  program  must  be  repeated  several  times  during  sequential  replacement 
of  SRAs.  However,  the  single  most  important  factor  in  the  dramatic  increases  in  EMT 
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was  lack  of  TPS  supplementary  data  needed  by  the  operator  to  discriminate  between  ATE, 
TPS  and  UUT  faults,  and  then  isolate  and  repair  them.  Such  documentation  was  available 
for  VAST  TPSs  but  was  not  delivered  for  RADCOM  TPSs  until  several  years  after  their 
introduction  to  the  fleet  (after  the  data  in  Table  ID  was  collected).  [Ref.  32] 

CASS  should  have  improved  ability  to  discriminate  between  station,  TPS 
and  UUT  faults,  resulting  in  decreased  EMT  for  the  same  UUTs.  It  is  designed  and 
warranted  to  continuously  monitor  station  and  TPS  inteiface  status  and  automatically 
indicate  all  faults  to  the  operator,  detecting  100%  of  the  faults  within  five  minutes  and 
isolating  them  to  a  single  CASS  SRA  within  15  minutes  95%  of  the  time  (as  detailed  in 
Chapter  III).  As  soon  as  a  CASS  fault  is  detected  that  affects  the  particular  test  being 
performed,  the  operator  may  disconnect  the  set  up  and  move  to  an  ’up’  station,  with  no 
additional  on-line  troubleshooting  required. 

However,  TPS  troubleshooting  time  has  historically  been  a  large 
contributor  to  EMT  [Ref.  32],  and  may  continue  to  be  a  problem  using  CASS,  depending 
on  the  comprehensiveness  of  TPS  self-test  capability  and  supplementary  data.  CASS 
TPSs  are  likely  to  be  more  reliable  and  maintainable  than  existing  TPS  for  the  same  UUT 
because  all  needed  instruments  are  intended  to  be  inside  the  station,  enabling  use  of 
simple  passive  IDs,  perhaps  containing  only  a  junction  box  and  cables.  But  there  are 
already  many  known  exceptions  to  use  of  passive  IDs.  For  example,  CASS  will  require 
a  ’power  head’  (an  active  ID  with  a  high  frequency  signal  processor)  for  UUTs  needing 
radio  frequencies  above  40  Ghz  because  it  was  technically  impractical  to  put  these 
instruments  in  the  station.  [Ref.  35]  Many  digital  WRAs  will  require  IDs 
containing  digital  multiplexers  while  many  electro-optical,  pneumatic  and  radar  UUTs  will 
require  IDs  with  liquid  coolant  and  extensive  mechanical  devices.  Designing 
comprehensive  self-testing  capability  into  such  IDs  may  be  prohibitively  expensive. 

Some  individuals  involved  in  TPS  development  believe  that  TPS 
performance,  reliability  and  maintainability  goals  are  not  affordable  for  a  large  percentage 
of  TPSs.  [Ref.  31]  NAVAIR  considers  TPS  development  an  area  of  high  risk  which  they 
are  attempting  to  control  through  development  of  a  standard  TPS  contracting  vehicle  to 
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be  used  for  each  of  the  thousands  of  TPS  to  be  developed.  The  technical  approach  to 
TPS  acquisition  now  being  considered  by  a  ’Red  Team’  of  NAVAIR  and  field  activity 
personnel  emphasizes  performance  specifications  calling  for  100%  fault  detection  and 
isolation  to  an  ambiguity  group  of  six,  95%  fault  isolation  to  an  ambiguity  group  of  two, 
an  MTBF  of  2500  hours  and  MTTR  of  60  minutes,  plus  a  tradeoff  analysis  to  optimize 
TPS  cost,  ID  complexity  and  PRT.  These  goals  have  been  achieved  for  already 
developed  support  of  support  (SOS)  TPS  for  CASS  station  components.  [Ref.  50] 

Another  critical  factor  affecting  TPS  reliability  and  maintainability  is  the 
way  TPS  developers  deal  with  new  instrumentation  requirements  for  emerging  weapon 
systems.  If  CASS  does  not  contain  all  needed  instruments,  they  may  be  added  to  the 
station  through  engineering  change  proposals  (ECPs)  or  pre-planned  product  improvement 
(P3)!,  or  they  may  be  placed  in  active  IDs  for  the  UUTs  that  need  the  function.  Many 
existing  ATE  systems  have  been  upgraded  to  accept  new  testing  requirements  using  active 
IDs  rather  than  station  modifications  because  it  is  often  cheaper  and  more  manageable  in 
terms  of  configuration  control.  Similar  reasoning  by  decision  makers  involved  in  CASS 
TPS  development  could  lead  eventually  to  fielding  of  many  complex  IDs  to  support  new 
weapon  systems. 

Consideration  of  each  of  these  factors  leads  to  the  conclusions  that  CASS 
station  self-testing  capability  will  tend  to  reduce  EMT  overall,  but  the  EMT  of  each  UUT 
will  be  highly  dependent  upon  the  tradeoffs  made  during  TPS  development  between  cost 
and  TPS  performance,  reliability,  maintainability  and  self- testability. 

b.  ATE  Availability  Effects  on  TAT 

Operational  availability  (A0)  is  the  probability  that  a  system,  when  used 
under  stated  conditions  in  an  actual  operational  environment,  will  operate  satisfactorily 
when  called  upon.  [Ref.  34:p.  65]  ATE  operational  availability  affects  the  LDT  portion 
of  TAT  through  its  effect  on  backlog. 

A  study  by  the  Center  for  Naval  Analysis  demonstrated  this  relationship 
between  operational  availability  and  TAT  (given  a  constant  number  of  ATE  stations  at 
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a  site)  for  four  ATE  stations:  VAST,  the  Avionics  Test  Set  (ATS),  the  Hybrid  Test  Set 
(HTS)  and  CAT  HID.  The  study  compared  the  actual  ATE  availability  and  repair  TAT 
achieved  by  these  stations  during  a  1987  USS  Constellation  deployment  to  the  TAT  that 
would  have  been  achieved  given  100%  ATE  availability.  The  likely  effects  of  100% 
ATE  availability  on  queue  length  and  TAT  were  determined  using  a  queuing  simulation 
called  the  Aviation  Logistics  Model  (ALM)  developed  by  CNA.  [Ref.  29] 

The  study  found  that  the  actual  average  TAT  for  VAST  was  9.7  days,  but 
fell  to  7.1  days  when  the  station  was  assumed  to  be  always  available.  For  ATS  it  fell 
from  4.8  days  actual  average  TAT  to  4.0  days  assuming  100%  availability.  Using  ASO 
manual  mles,  the  shorter  TAT  reduced  the  cost  of  AVCAL  spares  allowances  by  7.9% 
for  VAST  and  11%  for  ATS.  TAT  and  sparing  differences  turned  out  to  be  slight  for 
HTS  and  CAT  HID,  probably  because  they  had  greater  excess  capacity  than  VAST  and 
ATS  and  so  were  less  sensitive  to  changes  in  availability. 

It  is  important  to  note  that  even  very  low  levels  of  ATE  availability  may 
have  little  effect  on  TAT  if  a  site  has  sufficient  stations  to  compensate  for  it.  For  this 
reason,  if  all  else  is  held  constant,  improving  ATE  A0  will  either  reduce  the  number  of 
stations  required  or  reduce  TAT,  as  in  the  above  study.  (The  workload  model  used  to 
compute  station  requirements  at  a  site  based  on  a  given  Ac  and  other  key  input  variables 
is  considered  in  the  next  section.) 

To  make  comparison  of  A0  for  CASS  and  FOT  meaningful,  common 
assumptions  and  methodology  must  be  used  in  the  A„  calculation.  The  following 
paragraphs  establish  this  common  base  and  then  compare  the  computed  values  of  A0  for 
CASS  versus  three  members  of  the  FOT. 

A  critical  assumption  in  calculating  A„  is  what  constitutes  a  ’down’ 
station.  The  CNA  study  reported  above  avoided  this  definition  problem  by  using 
documented  downtime  of  individual  components  of  each  station  rather  than  A0  of  the 
entire  station.  Different  UUTs  require  different  station  ’building  blocks;’  some 
components  are  core  requirements  needed  for  testing  all  UUTs,  but  others  are  required 
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for  only  a  few  UUTs.  A  further  complication  is  that  a  station  component  may  be  partially 
down,  but  can  still  be  used  to  test  a  subset  of  UUTs. 

To  overcome  these  problems,  the  CASS  development  specification  for  A0 
is  based  on  the  assumption  that  any  failure  downs  the  entire  station.  Much  less 
conservative  (and  difficult  to  compare)  assumptions  are  commonly  used  in  calculating  A0 
for  members  of  the  FOT,  resulting  in  published  values  close  to  0.90.  [Ref.  36] 
The  1984  NAVAIR  methodology  used  for  calculating  the  targeted  CASS  AD  was  based 
on  the  standard  Navy  policy  for  calculating  the  \  of  weapon  and  support  systems. 
[Ref.  37]  It  used  the  following  equation:  [Ref.  38] 


A,  =  MTBF  ♦  k 

(MTBF  *  k)  +  MDTU 
and  k  =  X;. 


T„ 

where  Tc  =  Period  of  calendar  time  over  which  A„  of  the  system 
is  to  be  measured; 

T0  =  Operating  time  of  the  system  during  Tc; 
and  MDT„  =  Mean  unscheduled  downtime  (ignoring  scheduled  maintenance); 
=  MTTR  +  MLDT  +  MDT0; 

MTTR  =  Mean  time  to  repair; 

MLDT  =  Mean  logistic  delay  time; 

MDT0  =  Mean  downtime  for  other  delays  (outside  assistance, 
training,  documentation,  etc.). 


CASS  used  1984  VAST  data  as  the  baseline  for  estimating  many  of  these 
values.  The  k  factor  is  introduced  in  this  methodology  to  convert  MTBF  (which  assumes 
continuous  operating  time  except  during  repair)  into  units  of  continuous  elapsed  calendar 
time  (which  includes  those  time  periods  when  the  system  is  not  operating  and,  therefore, 
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will  not  fail)  so  that  all  elements  of  the  equation  will  be  in  common  units.  If  the  station 
is  always  in  active  use,  k  is  equal  to  1.  However,  when  there  are  long  periods  of  station 
inactivity  (because  the  ATE  shop  does  not  work  around  the  clock)  k  is  greater  than  1. 
It  equals  the  ratio  of  TJT0  which  represents  the  calendar  time  to  operating  time 
relationship  over  the  long  run  to  account  for  all  active  and  inactive  phases.  [Ref.  37:p. 
A- 13]  The  weighted  average  actual  operating  time  (T0)  for  all  VAST  sites  in  1984  was 
343.5  hours/station/month  (out  of  720  total  calendar  hours/month).  The  k  factor  for 
VAST  (and  assumed  for  CASS)  is  therefore: 


k  =  X;  =  720  hours/station/month  =  2.096. 

Tc  343.5  hours/station/month 

MLDT  for  VAST  was  80.9  hours.  To  account  for  the  initial  supply 
system  learning  curve  that  will  probably  increase  the  CASS  MLDT,  NAVAIR  increased 
this  value  by  50%,  to  120  hours. 

The  CASS  reliability  and  maintainability  specification  values  given  in 
Chaptf  r  HI  are  derived  from  inherent,  or  laboratory,  values  which  were  derated  to  reflect 
operational  values  expected  to  be  achieved  during  initial  fleet  introduction.  A  reliability 
derating  factor  of  2.7  was  chosen  on  the  basis  of  a  Rome  Air  Development  Center  study 
of  avionic  systems  which  determined  the  typical  ratios  of  MTBF/MTBFC  are  between  2.5 
and  2.7.  MTTR  was  doubled  to  account  for  the  initial  maintenance  learning  curve.  If  the 
follow  ng  terms  are  defined: 

MTBF,  =  inherent  MTBF; 

MTBF„  =  operational  MTBF; 

MTTR,  =  inherent  MTTR; 

MTTR0  =  operational  MTTR; 
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then 


MTBF0  =  MTBF,  =  400  hours  =  148  hours; 

2.7  2.7 

MTTR,  =  MTTR,  *2  =  1  hour  *2  =  2  hours. 

Using  an  assumed  value  for  MDTe  of  10  hours,  CASS  A0  is  calculated 

as  follows: 

Ac  = _ MTBF  *  k _ 

(MTBF  *  k)  +  MTTR0  +  MLDT  +  MDTC 

= _ (148  *  2.0961 _ =  0.70. 

(148  *  2.096)  +  2+  120  +10 


Sensitivity  analysis  demonstrated  that  the  three  principal  drivers  of  Ac  are 
Tc,  MLDT  and  MTBF;  a  10%  change  in  any  of  one  these  parameters  (up  or  down)  results 
in  only  a  3%  change  in  Aa.  This  is  an  indication  of  the  sensitivity  of  the  results  to  errors 
in  estimating  the  input  values.  It  is  notable  that  A„  is  not  overly  sensitive  to  small  errors 
in  individual  input  values,  but  even  small  errors  in  multiple  input  values  are  cumulative 
and  can  strongly  affect  the  A0  prediction. 

NAVAIR  lowered  the  contractual  value  for  A„  from  0.70  to  0.68  because 
the  latter  is  the  CNO  goal  for  weapon  system  availability.  To  provide  adequate  support 
for  these  weapon  systems  CASS  should  be  able  to  achieve  at  least  this  level  of 
availability.  [Ref.  39] 

Table  V  shows  the  estimated  \  and  its  components  for  CASS  VAST, 
CAT  HID  and  RADCOM  calculated  using  the  above  methodology.  A  value  for  k  of 
2.096  and  MDT0  of  10  hours  was  assumed  for  all  four  ATE  systems.  All  other  FOT 
input  data  was  obtained  from  the  Naval  Air  Engineering  Center  which  compiled  it  by 
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validating  and  cleaning  up  the  notoriously  inaccurate  3M  data  base.  They  used  a 
representative  sample  of  stations  from  sea  and  shore  sites,  including  31  VAST  stations, 
16  RADCOMs,  and  19  CAT  mDs,  for  the  period  April  1985  to  February  1990.  (Note 
that  the  data  for  VAST  in  Table  V  differs  from  the  1984  VAST  data  used  in  the  above 
analysis.)  {Ref.  40] 

Table  V.  Comparison  of  Ac  for  CASS  vs  Members  of  the  Family  of  Testers. 


Parameter 

VAST 

CAT  HID 

RADCOM 

CASS 

MTBF 

122 

457 

341 

148 

MTTR 

1.782 

6.92 

7.917 

2.0 

MLDT 

200 

655 

744 

120 

MDT0 

10 

10 

10 

10 

A) 

.547 

.588 

.484 

.70 

(All  tunes  are 

given  in  hours, 

The  comparison  indicates  that  if  NAVAIR  assumptions  are  correct,  upon 
initial  fleet  introduction  CASS  may  be  less  reliable  than  RADCOM  and  CAT  HID,  but 
more  reliable  than  VAST  (indicated  by  MTBF  for  each  system).  Its  reliability  is  expected 
to  grow  during  its  initial  ’bum-in’  period,  however.  CASS  may  be  far  more  maintainable 
than  RADCOM  and  CAT  HID  and  about  the  same  as  VAST  (indicated  by  MTl'K). 
Overall,  CASS  A0  is  significantly  higher  than  the  A0  of  the  three  FOT  stations,  given  the 
assumptions  discussed  above. 

The  CASS  assumption  for  MLDT  of  120  hours  is  most  open  to  challenge. 
It  is  much  lower  than  the  MLDT  for  any  of  the  FOT  systems,  even  though  CASS  must 
use  the  same  supply  system  and  faces  many  of  the  same  logistic  delays.  If  the  CAT  HID 
MLDT  value  of  655  hours  is  substituted  in  the  equation  for  the  assumed  CASS  MLDT 
of  120  hours,  CASS  A0  drops  from  0.70  to  0.317.  Even  using  the  current  VAST  MLDT 
(which  at  200  hours  is  significantly  better  than  CAT  HID  MLDT)  the  CASS  A„  would 
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be  0.59,  which  is  only  a  slightly  higher  value  for  A„  than  the  three  FOT  systems.  The 
conclusion  drawn  from  this  comparison  is  that  CASS  availability  will  not  be  better  than 
the  availability  of  existing  ATE  unless  its  MLDT  is  significantly  lower  than  that  of  the 
three  FOT  systems. 

c.  Influences  on  ATE  Supportability 

RADCOM  and  CAT  HID  have  much  higher  MTTR  and  MLDT  than 
VAST  because  changes  in  the  architecture  of  these  newer  testers  limit  their  ability  to  be 
organically  repaired  and  spares  allowances  have  not  adequately  covered  the  long  TAT  for 
commercial  depot  repair. 

The  architectural  approach  used  in  most  ATE  developed  since  VAST 
employs  predominately  off-the-shelf  (non-developmental)  hardware  to  ’rack  and  stack’ 
the  needed  functional  components.  As  much  as  80%  of  the  circuitry  is  typically  made 
up  of  embedded  commercial  test  equipment  (CTE).  Testers  may  be  assembled  from 
different  combinations  of  about  40  standard  commercial  instruments  (such  as  RF 
generators,  vector  voltmeters  and  signal  analyzers)  to  provide  input  signals  and  response 
measurement.  These  are  joined  with  a  station  computer,  software  and  a  small  set  of 
contractor  developed  equipment  (CDE)  to  provide  power  and  functions  such  as  interface 
with  the  ID  and  the  operator.  [Ref.  13:p.  5-6] 

The  advantages  of  an  ATE  acquisition  strategy  emphasizing  CTE  are  that 
it  is  the  fastest  way  to  build  test  equipment  and  it  has  low  up  front  costs  and 
technical/schedule  risks  compared  to  using  CDE  as  was  the  case  in  the  older  VAST. 
Most  CTE  have  proven  high  levels  of  reliability  in  commercial  applications  and  most 
have  performed  similarly  well  in  ATE,  as  reflected  in  the  relatively  high  levels  of 
reliability  of  RADCOM  and  CAT  IIID  shown  above. 

This  acquisition  strategy  causes  many  supportability  problems,  however, 
including  reliance  on  primarily  non-organic  depot  maintenance  and  short  obsolescence 
cycles.  The  phased  support  concept  has  also  induced  supportability  problems  during 
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introduction  of  new  ATE.  The  following  paragraphs  explain  these  problems  and  compare 
the  alternative  approaches  used  in  CASS  development. 

Intermediate  level  maintenance  support  for  testers  employing  CTE  is  very 
limited  because  NAVAIR  policy  restricts  maintenance  of  all  embedded  CTE  to  depot 
repair  activities.  This  decision  was  made  in  1987  in  an  effort  to  improve  configuration 
control  of  commercial  instrumeiits.  CTE  manufacturers  continually  update  their  products 
by  discontinuing  product  lines  (and  their  repair  parts)  and  by  making  frequent  design 
changes  in  current  production  instruments  to  improve  them  and  keep  up  with  state  of  the 
art.  While  many  companies  such  as  Hewlett  Packard  (used  extensively  in  ATE)  pledge 
to  support  their  product  for  seven  years,  it  is  nonetheless  difficult  for  Navy  repair  sites 
to  maintain  the  right  spare  parts  and  publications,  especially  at  intermediate  level. 
[Ref.  41] 

Despite  this  policy,  experienced  I  level  maintenance  technicians  or 
technical  representatives  often  perform  needed  repairs  to  CTE.  When  they  cannot, 
technicians  must  ship  the  CTE  to  the  depot  even  when  only  minor  adjustments  are 
required.  This  has  seriously  affected  RADCOM  and  CAT  HID  MLDT  and  availability. 
Due  to  a  commercial  depot  repair  backlog  approaching  two  years  for  the  CTE  in  these 
testers,  there  were  over  600  parts  awaiting  induction  by  Grumman  as  of  23  January  1990. 
[Ref.  42]  Of  these,  119  were  holding  stations  down.  Much  of  this  delay  arose  during  the 
transition  from  prime  contractor  support  by  Grumman  to  Navy  support  after  MSD  in 
1988,  which  occurred  without  adequate  spares  on  hand  to  meet  the  demand.  A  snapshot 
of  the  resulting  Atlantic  fleet  ATE  status  on  26  January  1990  reveals  that  9  of  25 
RADCOM  stations  and  8  of  27  CAT  HID  stations  were  down  for  parts  ordered  from  off 
the  ship  or  station.  With  one  third  of  fleet  RADCOM/CAT  EUD  assets  typically  down 
(as  on  this  day),  type  commanders  have  kept  deployed  carrier  ATE  stations  up  by 
cannibalizing  station  parts  from  shore  AIMDs  and  pierside  carriers.  [Ref.  43] 

The  short  obsolescence  cycles  of  CTE  (averaging  seven  to  fourteen  years) 
also  creates  ATE  support  ability  problems.  Because  the  government  generally  does  not 
buy  level  III  engineering  drawings  for  these  instruments,  spare  parts  procurement  becomes 


69 


costly  after  the  contractor  stops  supporting  them.  For  example,  29  repairable  RADCOM 
circuit  cards  made  by  HP  are  currently  obsolete  and  must  be  reprocured  in  FY-91  from 
other  sources  using  inadequate  level  II  drawings.  These  parts  will  require  significant 
reverse  engineering  to  attain  the  same  form,  fit  and  function  as  the  original  parts.  The 
redesign  may  also  significantly  change  their  repair  processes.  [Ref.  44] 

Use  of  phased  support  plans  is  at  the  root  of  many  supportability 
problems  associated  with  the  introduction  of  new  ATE  systems.  For  example,  the 
RADCOM  IOC  date  was  September  1982  and  the  introduction  is  now  almost  complete, 
yet  many  RADCOM  publications  were  not  ordered  from  Grumman  until  1986  and  many 
have  not  been  delivered.  [Ref.  44]  Maintenance  Assist  Modules  (MAMS)  needed  for 
station  troubleshooting  were  not  delivered  until  1988.  Many  operator  and  maintainer 
courses  were  overloaded  until  1989  so,  until  recently,  less  than  half  of  the  RADCOM 
operators  at  many  shore  sites  were  formally  trained.  Sea  and  shore  AIMDs  experienced 
significant  problems  supporting  their  stations  (reflected  in  the  MTTR,  MLDT  and  A0  data 
above)  during  the  several  years’  wait  for  the  logistic  ’tail’  to  catch  up. 

The  negative  consequences  of  a  phased  support  concept  include:  [Ref. 

11] 

•  ATE  systems  must  often  be  nurtured  and  micro-managed  until  all  logistic  elements 
are  in  place. 

•  Support  of  ATE  cannot  be  tested  during  development,  so  poor  design  for  logistic 
supportability  may  be  discovered  only  by  fleet  users. 

•  Few  incentives  exist  for  the  contractor  to  design  for  minimum  life  cycle  cost  and 
adequately  perform  logistic  support  analysis  (LSA). 

•  Few  incentives  exist  to  pay  for  a  logistic  support  tail  once  a  system  is  operating 
because  ATE  deficiencies  can  be  compensated  for  by  higher  expenditures  for 
weapon  system  spares  and  depot  repairs  (which  may  increase  total  life  cycle  costs). 

CASS  is  likely  to  eliminate  most  of  these  supportability  problems 
affecting  existing  ATE  systems  for  a  number  of  reasons.  Most  ;mportantly,  CASS  will 
have  no  phased  support.  All  logistic  elements  including  spares  are  under  development 
concurrently  with  the  prime  equipment,  will  be  provided  organically,  will  be  delivered  up 
front,  and  will  be  tested  during  TECHEVAL  and  OPEVAL.  Second,  maintainability  will 
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be  improved  by  the  high  degree  of  commonality  between  tester  configurations  and  by  the 
’function  on  a  card’  approach  that  eliminates  most  CTE  instruments.  It  should  have  faster 
repairs  due  to  its  extensive  self-testability,  modularity,  automated  technical  information 
system  and  embedded  training  capability.  The  MTTR  is  warranted  not  to  exceed  2.0 
hours,  compared  to  the  6.9  and  7.8  hours  for  CAT  HDD  and  RADCOM.  It  has  a  power 
conditioning  system  and  battery  backup  sufficient  to  overcome  many  of  the  reliability 
problems  experienced  by  current  testers  due  to  power  transients. 

There  are,  however,  several  areas  of  technical  risk  that  could  impact 
supportability.  Perhaps  the  most  significant  is  the  high  forced  air  cooling  requirement  of 
CASS  which  exceeds  the  current  capabilities  of  carriers  to  provide.  An  adequate  and 
reliable  cooling  system  is  of  vital  importance  to  ensure  CASS  is  more  supportable  than 
existing  systems.  The  current  plan  to  solve  this  problem  is  to  deliver  a  portable  air  chiller 
along  with  each  CASS  station.  This  chiller  will  require  a  water  line,  power  and  an 
additional  four  square  feet  of  deck  space.  [Ref.  35]  The  supportability  issues  are,  first, 
that  the  chiller  must  also  have  organic  logistic  support,  and  second,  that  because  it  will 
form  a  series  system  with  CASS  it  will  reduce  total  reliability  of  the  test  system  by  a 
factor  equal  to  the  chiller’s  reliability.  The  chiller  may  be  more  reliable  than  facility  air 
conditioning  on  which  ATE  now  depends,  but  it  is  sure  to  increase  CASS’  demand  for 
electrical  power  which  is  constrained  by  the  inability  of  ships  to  meet  ever  growing 
requirements.  The  significance  of  these  power  and  air  problems  is  illustrated  by  a  recent 
Coral  Sea  cruise  in  which  RADCOM  technicians  reported  an  average  of  1.6  hours  down 
each  day  due  to  facility  power  and  cooling  problems.  [Ref.  45] 

In  the  event  that  CASS  overheats,  among  the  first  components  likely  to 
be  damaged  is  the  RF  subsystem,  which  contains  the  few  items  of  Cl  E  in  CASS  (other 
than  the  VAX  computer).  They  wiM  not  be  organically  supported,  and  the  government 
has  no  level  HI  drawings.  They  are  an  improvement  over  present  CTE  in  terms  of 
supportability,  however,  because  they  are  modular  cards  rather  than  complete  instruments 
and  the  manufacturer  (HP)  has  contracted  to  provide  support  for  20  years.  [Ref.  35] 
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The  key  to  reducing  MLDT  from  the  600+  hours  experienced  by 
RADCOM  and  CAT  IIID  to  the  120  hours  targeted  for  CASS  is  to  have  up  front  delivery 
of  organic  intermediate  and  depot  level  support.  This  will  significantly  reduce  the  repair 
TAT  of  CASS  SRAs  in  comparison  to  the  TAT  of  SRAs  for  non-organically  supported 
testers.  Adequate  spares  must  also  be  stocked.  To  achieve  the  CASS  goal  for  A0  of  0.68, 
NAVAIR  has  therefore  set  the  policy  that  A,,  will  drive  CASS  SRA  sparing.  The 
Aviation  Supply  Office  (ASO)  will  use  a  ’readiness  based  sparing  model’  known  as 
’ARROWS’  to  compute  AVCAL/SHORECAL  allowances  for  a  site.  CASS  is  the  first 
support  equipment  system  to  use  this  model  (until  now  ARROWS  has  been  used  only  for 
weapon  systems).  If  it  works  as  intended,  sufficient  spares  will  be  stocked  to  achieve  the 
targeted  A„  value  of  0.68  given  CASS’  forecast  reliability.  [Ref.  39] 

A  fundamental  problem  remains  unsolved,  however,  because  there  is  still 
no  adequate  measure  of  availability  that  accounts  for  partially  capable  CASS  stations  that 
can  test  some  UUTs  and  not  others.  Achieving  an  A0  of  0.68  does  not  mean  the  station 
has  a  68%  probability  of  being  able  to  perform  satisfactorily  when  called  upon.  Because 
NAVAIR  assumes  that  any  fault  (however  minor)  downs  the  station,  its  probability  of 
being  able  to  perform  satisfactorily  (able  to  test  some  but  not  all  UUTs)  may  be 
considerably  higher  than  its  measured  value  of  A^,. 

4.  Workload  Analysis 

The  number  and  type  of  ATE  stations  delivered  to  a  site  are  the  final  key 
determinate  of  TAT.  Providing  too  few  ATE  stations  to  a  site  results  in  unplanned  levels 
of  backlog  (which  is  a  principal  cause  of  weapon  system  LDT)  and  drives  up 
AVCAL/SHORECAL  allowances.  An  accurate  workload  model,  on  the  other  hand, 
facilitates  tradeoff  analysis  between  cost  of  ATE  and  cost  of  weapon  system  spares  to 
achieve  a  given  level  of  customer  service  and  enables  decision  makers  to  compensate  for 
low  ATE  \  or  long  EMT  of  individual  UUTs. 
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Inputs  which  must  be  considered  in  workload  analysis  include: 

•  UUT  induction  rates; 

•  EMT  of  each  UUT; 

•  Facility  operating  hours; 

•  Station  A^; 

•  Desired  ATE  utilization  rate; 

•  Desired  level  of  customer  service; 

•  Queue  discipline; 

To  adequately  account  for  these  factors  in  determining  the  correct  number  of 
stations  to  install  at  a  site,  a  queuing  simulation  should  be  used.  The  CASS  System 
Synthesis  Model  (SSM)  contains  such  a  model  (as  discussed  in  Chapter  III)  but  it  is  still 
under  development.  Once  accepted  by  the  government  from  GE  it  will  be  used  to 
compute  CASS  station  requirements  at  each  site.  Station  requirements  up  until  now  have 
been  computed  with  a  linear  algebraic  workload  model  which  uses  average  values  for 
inherently  random  parameters  such  as  EMT  and  UUT  inductions.  It  attempts  to  account 
for  the  effects  on  a  repair  queue  of  random  surges  in  UUT  inductions  and  variations  in 
EMT  by  setting  the  CASS  station  utilization  to  some  value  less  than  one. 
[Ref.  46]  A  similar  model  has  been  used  for  determining  requirements  for 
earlier  ATE  systems  without  benefit  of  a  far  more  accurate  queuing  simulation  (as  CASS 
should  eventually  have).  As  a  result,  too  few  ATE  stations  have  been  procured  for  many 
members  of  the  FOT  to  achieve  the  desired  level  of  customer  service  or  minimize  total 
weapon  system  support  costs.  The  early  problems  with  VAST  workload  are  a  typical 
example  of  this,  as  discussed  in  the  summary  of  ATE  evolution  in  Chapter  II.  RADCOM 
is  another  example,  as  will  be  demonstrated  below. 

This  section  compares  two  studies  of  RADCOM  workload  to  illustrate  the 
effects  of  different  assumptions  and  input  values  on  the  linear  algebraic  model’s 
calculation  of  the  number  of  stations  required.  It  also  indicates  the  degree  of  risk  the 
CASS  introduction  team  faces  as  they  forecast  CASS  station  requirements  using  this 
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model  and  predicted  input  values.  The  CASS  SSM  model  is  reviewed  in  more  detail  at 
the  end  of  this  section. 


The  basic  linear  algebraic  model  used  for  RADCOM  workload  analysis  is  as 


follows: 


Stations  required  =  TOT  WKLD  HRS  REQ: 

StAV 

TOT  WKLD  HRS  REQ  =  WKLD/M  summed  over  all  UUTj  in  each  aircraft, 

for  each  site; 

PFH/M,  *  WKLD/FLTH^; 

SLF  *  A0  *HRWD  *  DPM; 


WKLD/Mjj 

StAV 

where 

TOT  WKLD  HRS  REQ 

WKLD/M,  j 

PFH/M, 

WKLD/FLTH,j 

StAV 

SLF 

HRWD 

DPM 


Total  expected  workload  hours  required  per 
month  for  the  site; 

Expected  workload/month  for  the  i,h  aircraft 
and  the  j,h  UUT; 

Peak  flight  hours/month  (highest  flight  hours 
expected  in  any  month)  for  the  i,h  aircraft; 
Total  expected  workload/flight  hour  for  the 
i'h  aircraft  and  j'h  L^TT; 

Station  availability; 

Surge  loading  factor; 

Hours  of  work/day; 

Days/month. 


WKLD/FLTH  is  a  unitless  value  calculated  as  the  sum  of  EMT/MTBF  for  each 
UUTj  in  each  aircraft,  to  be  tested  on  the  ATE  system.  For  example,  the  radar  data 
converter  for  the  A-6E  aircraft  has  an  MTBF  of  39  hours  and  EMT  of  4.19  hours. 
WKLD/FLTH  for  this  WRA  is  thu.  4  ’9/39,  or  0.10743.  [Ref.  47]  This  value 
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is  computed  for  the  remained  ATE-supported  A-6E  UUTs  to  obtain  a  total  A-6E 
WKLD/FLTH.  This  is  added  to  the  totals  computed  similarly  for  the  other  aircraft  types 
to  obtain  the  overall  WKLD/FLTH  for  the  site. 

The  surge  loading  factor  is  equivalent  to  the  desired  station  utilization  rate. 
It  is  less  than  1.0  to  protect  against  surges  in  operating  tempo  and  the  effects  of 
randomness  in  the  distributions  of  inductions  and  EMT  which  could  cause  undesired 
levels  of  backlog  if  a  station  is  loaded  to  full  capacity. 

This  model  was  used  in  RADCOM  workload  studies  conducted  by  the  Pacific 
Missile  Test  Center  (PMTC)  [Ref.  47]  and  by  Grumman  Aerospace  Corporation  (under 
NAVAIR  contract).  [Ref.  48]  It  is  also  the  basis  for  the  workload  module  of 
the  CASS  SSM  developed  for  NAVAIR  by  General  Electric.  Although  the  three 
organizations  called  some  of  the  variables  by  different  names,  the  models  they  used  are 
all  essentially  as  shown  above  (which  is  based  on  the  PMTC  study).  The  following 
paragraphs  compare  the  results  and  input  values  of  the  1985  Grumman  study  (which 
considered  E-2C  RADCOM  support  requirements)  and  the  1989  PMTC  study  of  the  full 
RADCOM  workload. 

The  Grumman  study  underestimated  EMT  and  overestimated  station  A0  in 
p'.Jicting  site  requirements  for  RADCOM  stations  to  support  the  E-2C.  PMTC 
determined  that  150  RADCOM  stations  are  needed  Navy  wide,  compared  to  the  122  that 
were  procured  based  on  this  Grumman  analysis  (as  well  as  other  similar  Grumman  studies 
of  RADCOM  quantities  needed  to  support  the  A-6E  and  EA-6B).  Even  before  the  PMTC 
study,  NAVAIR  (AIR-552)  recognized  that  the  first  calculation  of  site  requirements  was 
too  low  and  in  1986  delivered  an  extra  station  for  many  sea  and  shore  AIMDs.  [Ref.  43] 
In  1988  COMNAVAIRLANT  and  COMNAVAIRPAC  urged  that  quantities  be  further 
increased  above  planned  deliveries  in  order  to  reduce  backlog,  recommending  quantities 
at  major  sea  -nd  shore  AIMDs  very  close  to  PMTC’s  later  calculations.  [Ref.  43] 

Grumman  determined  the  values  of  EMT  for  each  UUT  based  on  1981-1982 
3M  data  compiled  very  early  in  the  RADCOM  program.  EMT  for  new  UUTs  never 
before  supported  were  based  on  estimates  by  systems  engineers,  using  the  relative 
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complexity  of  components  and  comparisons  to  similar  systems.  The  EMT  values  for 
UUTs  offloaded  from  VAST  were  reduced  by  20%  based  on  the  expectation  that 
RADCOM  would  reduce  VAST  EMT  by  that  amount  (an  expectation  which  was  not 
realized,  as  shown  by  the  comparison  of  VAST  and  RADCOM  EMT  earlier  in  this 
chapter).  From  these  EMT  values  the  study  calculated  a  mean  station  workload/flight 
hour  (WKLD/FLTH)  of  0.619  using  the  method  shown  above. 

PMTC  recalculated  this  value  and  found  it  to  be  0.99884  after  RADCOM  had 
been  in  the  fleet  for  several  years  and  more  accurate  and  complete  3M  data  on  EMT  was 
available.  The  increase  in  station  WKLD/FLTH,  with  everything  else  held  constant, 
results  in  a  61%  increase  in  RADCOM  station  requirements  to  support  the  E-2C  aircraft. 

The  Grumman  overestimate  of  RADCOM  A0  (contained  in  the  denominator 
of  the  above  equation)  appears  in  a  factor  they  called  "workload  efficiency,"  which  is 
equivalent  to  the  portion  of  the  model  PMTC  labeled  "A,,  *  SLF."  Whereas  Grumman 
used  0.8  for  this  value,  PMTC  calculated  the  value  based  on  a  more  complex  analysis 
which  used  the  values  of  A„  and  SLF  for  various  types  of  sites.  These  values  are  shown 
in  Table  VI. 

The  \  values  of  0.746,  0.777  and  .842  for  carriers,  shore  AIMDs  and  depots, 
respectively,  were  computed  from  3M  data.  The  SLF  values  were  derived  from  a 
multiple  server  queuing  model  that  attempted  to  account  for  the  varying  levels  of  surge 
protection  needed  at  sites  with  different  numbers  of  ATE  stations  to  achieve  the  same 
level  of  customer  service.  Queuing  theory  predicts  that  at  a  site  with  two  stations,  UUTs 
will  have  a  significantly  lower  waiting  time  to  be  serviced  than  if  there  is  one  station, 
while  each  additional  station  installed  at  the  site  reduces  waiting  time  by  an  ever 
decreasing  amount.  PMTC  made  a  ’first  pass’  estimate  of  station  requirements  for  each 
site  and  used  these  values  in  the  queuing  analysis  to  compute  the  SLF.  For  example, 
large  shore  AIMDs  (such  as  AIMD  Oceana)  which  are  assumed  to  need  four  RADCOM 
stations  have  an  SLF  equal  to  0.78.  Shore  AIMDs  assumed  to  need  only  two  stations  to 
support  their  smaller  workload  need  greater  surge  protection;  their  SLF  therefore  drops 
to  0.72.  The  workload  efficiency  for  a  large  shore  AIMD  is  thus  computed  as: 
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Table  VI.  PMTC  Calculation  of  RADCOM  Woikload  Efficiency. 


Type  of 

Site 

Station 

Qty 

A0 

SLF 

(from 

Station 

Qty) 

Workload 
Efficiency 
(A*  *  SUP) 

Carrier  AIMD 

4 

.746 

.78 

58 

Shore  AIMD 

4 

.777 

.78 

.56 

Shore  AIMD 

1 

.777 

.62 

.48 

Depot 

2 

.842 

.72 

.61 

Depot 

1 

.842 

.62 

.52 

Workload  Efficiency  =  A0  *  SLF 

=  0.777  *  0.78  =  0.56. 

The  workload  efficiency  factor  for  the  other  types  of  sites  were  computed  similarly.  This 
process  is  iterative  because  the  quantity  of  stations  finally  calculated  using  the  model 
must  be  the  same  as  the  first  pass  estimate  that  feeds  into  it. 

It  is  not  at  all  clear  what  the  ’correct’  value  is  for  A„  given  the  difficulties  in 
defining  its  meaning,  but  the  PMTC  results  (based  on  the  higher  EMT  and  lower  values 
for  Ac  *  SLF)  come  nearer  to  reflecting  actual  station  quantity  requirements.  The  model 
is  very  sensitive  not  only  to  errors  in  predicting  values  such  as  A„  and  EMT,  but  to 
assumptions  concerning  the  surge  loading  factor  and  number  of  hours  worked  per  day. 
When  PMTC  recalculated  total  station  requirements  using  a  surge  loading  factor  of  1 .0 
(100%  utilization)  for  all  sites,  total  RADCOM  station  requirements  dropped  from  150 
to  125.  This  quantity  is  not  significantly  higher  than  the  122  that  were  actually  procured. 
Station  requirements  dropped  from  150  to  127  when  PMTC  assumed  that  all  AIMDs  work 
24  hours  per  day  rather  than  the  more  typical  16  hour  workday  ashore  and  24  hour 
workday  at  sea. 
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The  Naval  Air  Engineering  Center,  which  manages  the  CASS  SSM  model, 
confirmed  the  sensitivity  of  their  version  of  the  model  to  EMT,  A^  and  surge  loading 
factor.  They  ran  a  sensitivity  analysis  of  the  workload  model  of  the  CASS  SSM  at  the 
author’s  request  for  this  thesis  research,  using  assumed  input  values  and  six  hypothetical 
UUTs.  The  results  show  that  doubling  the  average  EMT  doubles  the  quantity  of  stations 
required  at  a  site.  An  18%  decrease  in  either  or  the  surge  loading  factor  (which  the 
SSM  calls  "desired  station  utilization")  produce  18%  increases  in  the  number  of  stations 
required.  Thus  it  is  simple  to  predict  changes  in  this  algebraic  model  output  as  inputs  are 
varied  because  they  are  linearly  related.  They  are  not  so  simply  related  in  a  queuing 
model.  [Ref.  49] 

These  relationships  demonstrate  the  implications  of  errors  in  assumptions  and 
predictions  for  the  CASS  introduction.  If  too  few  CASS  stations  are  initially  delivered 
to  handle  the  assigned  testing  load,  as  occurred  in  both  the  RADCOM  and  VAST 
programs,  the  results  may  be  unplanned  backlogs,  a  reduction  in  the  level  of  customer 
service  provided  by  the  AVCAL/SHORECAL  (at  least  temporarily  until  allowances  are 
raised),  and  a  disruption  of  the  introduction/transition  schedule.  [Ref.  50] 

This  schedule  disruption  may  occur  if,  first,  NAVAIR  must  redistribute  CASS 
stations  to  increase  their  allowances  at  critical  sites,  or  second,  NAVAIR  must  retain  FOT 
testers  at  some  sites  as  backups  to  CASS  even  after  their  TPSs  are  offloaded.  As  an 
example  of  the  latter  response  to  overloaded  ATE  stations,  the  type  commanders  and  A-6 
Assistant  Program  Manager  for  Logistics  (APML)  decided  to  retain  Mini-SACE  stations 
(used  to  support  the  A-6E)  aboard  some  carriers  (such  as  Ranger)  as  backups  after  Mini- 
SACE  UUTs  were  supposed  to  have  completed  transition  to  RADCOM  because  there  are 
not  enough  RADCOMs  in  place  to  handle  the  full  testing  load.  [Ref.  51]  and 
[Ref.  52] 

The  task  of  estimating  CASS  station  requirements  is  far  more  complex  than 
the  introduction  of  earlier  ATE  systems  because  the  program  has  a  significantly  larger 
scope  and  because  the  high  degree  of  commonality  between  CASS  stations  permits  UUTs 
to  be  run  on  several  different  station  configurations. 
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As  noted  in  Chapter  ID,  the  SSM  was  developed  to  optimize  the  number  and 
mix  of  stations  at  a  particular  site,  using  mapping,  workload  and  simulation  models. 
Figure  12  shows  the  process  used  for  determining  an  optimized  site  profile  using  the 
model.  [Ref.  46]  The  workload  (as  shown  in  the  first  block)  is  determined  by  first 
reading  from  a  computer  data  base  the  weapon  system  complement  of  a  site,  the  WRAs 
in  each  weapon  system,  and  the  SRAs  in  each  WRA.  Induction  rates  are  calculated  using 
the  quantity  and  MTBF  of  each  WRA/SRA  and  the  appropriate  weapon  system  planning 
documents  (WSPDs)  containing  projected  combat/peacetime  operational  hours  per 
platform.  The  output  of  this  process  is  the  potential  CASS  workload  at  the  site  if  it  is 
able  to  test  all  WRAs/SRAs.  The  Mapping  model  is  used  to  perform  UUT-to-station 
mapping  to  indicate  which  UUTs  may  be  tested  by  which  station  configurations  (and  also 
which  UUTs  cannot  be  tested  by  CASS). 

The  second  and  remaining  steps  shown  in  the  figure  are  performed  using  the 
workload  model  in  conjunction  with  a  linear  programming  model  that  considers  all 
possible  UUT  to  station  type  assignments  and  minimizes  the  unconstrained  objective 
function: 


where 


Z  =  sum  (rank(i)  *  quantity(i)); 


i  =  index  to  each  station  type  required; 
rank  =  user  specified  desirability  rank  for  each  station  configuration; 
quantity  =  number  of  each  station  type. 


The  desirability  rank  is  based  on  either  the  relative  cost  or  space  requirements 
of  each  station  type.  For  example,  the  hybrid  configuration  is  the  cheapest  and  the 
smallest  (because  it  has  no  sixth  rack),  so  will  have  the  highest  rank  (smallest  number). 
The  requirements  of  a  specific  site  will  dictate  whether  space  or  cost  is  the  most 
important  ranking  criterion. 
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the  System  Synthesis  Model  (SSM). 
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The  minimum  quantity  of  each  type  required  at  a  site  (shown  in  the  second 

block  of  the  figure)  is  computed  using  the  workload  model.  Inputs  include  the  A0  and 

desired  utilization  for  each  station  configuration  plus  the  induction  rates  and  EMT  of  all 

UUTs  which  can  be  tested  on  that  station  type  only.  The  maximum  number  of  each 

station  type  (shown  in  the  third  block)  is  computed  similarly,  but  considers  the  induction 

rates  and  EMT  of  all  UUTs  that  may  feasibly  be  tested  on  that  station  type.  Table  VII 

is  a  hypothetical  example  of  minimum  and  maximum  UUT  maps  showing  three  CASS 

configurations  with  different  sets  of  UUTs  that  may  be  mapped  to  them.  For  example, 

the  CNI  station  is  the  only  configuration  that  will  test  UUTs  1,  2,  and  7  (which  sets 

minimum  CNI  station  quantities)  but  it  will  also  test  UUTs  4,  5,  6,  and  10  (which  sets 

maximum  CNI  station  quantities  if  these  UUTs  are  included  in  its  workload).  The  actual 

number  of  CNI  stations  placed  at  a  site  will  be  some  quantity  between  these  values 

determined  by  the  linear  programming  model. 

Table  VII.  System  Synthesis  Model  Mapping  of  UUTs  to  CASS  Station 
Configurations 


CASS 

Configuration 

UUTs  with  a  Single 
Station  Map 

Total  UUTs  Mapped  to 
Each  Station 

CNI 

1,2,7 

1,2,4,5,6,7,10 

Radar 

3,8 

3,4,6,8,10 

DDS 

9 

5,6,9 

All  possible  combinations,  or  profiles,  of  station  configurations  are  determined 
using  this  process,  including  their  resulting  CASS  self-repair  SRA  inductions.  Those 
profiles  which  cannot  meet  the  total  workload  requirements  (determined  by  the  workload 
model)  are  eliminated  from  consideration.  The  linear  programming  model  then  calculates 
the  objective  function  of  each  remaining  profile,  ranks  them  by  their  level  of  merit  and 
prints  them  as  the  final  output  of  the  SSM.  The  profiles  with  the  lowest  level  of  merit 


(there  will  likely  be  ties)  are  optimal.  [Ref.  46:p.  44] 
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The  CASS  SSM  model  is  still  under  development  and  has  not  been  used 
extensively  for  determining  site  requirements  in  the  initial  introduction  plan.  The  queuing 
simulation  model  must  ultimately  be  integrated  with  the  mapping  and  LP  models  before 
site  requirements  can  be  determined  with  a  high  degree  of  confidence  in  the  results.  It 
will  likely  be  used  in  place  of  the  linear  workload  model  for  determining  minimum, 
maximum  and  optimal  station  profiles  following  the  process  shown  in  Figure  12.  The 
identities  of  UUTs  to  be  tested  on  CASS  at  each  site  must  also  be  determined,  as  well  as 
accurate  values  for  their  EMT.  The  SSM  may  be  used  at  the  current  stage  of  planning 
only  for  estimating  gross  station  requirements  suitable  for  scheduling  funding 
requirements. 

Until  input  data  is  more  accurate  and  the  simulation  model  is  ready,  NAVAIR 
is  attempting  to  err  on  the  side  of  scheduling  too  many  stations  at  a  site.  Nevertheless, 
a  low  initial  guess  of  station  requirements  made  several  years  ago  resulted  in  inclusion 
of  too  few  stations  in  the  Program  Objectives  Memorandum  (POM)  and  a  shortfall  in 
station  procurement  hinds  through  FY-94  totaling  $543  million.  [Ref.  50]  To  make  up 
for  it,  NAVAIR  is  now  considering  several  plans,  including: 

•  Slipping  the  transition  schedule  of  selected  programs.  Many  programs  that  are 
critical  to  the  introduction  of  CASS  aboard  CVs  are  protected  from  slippage, 
however,  including  the  offload  of  existing  testers  such  as  VAST,  AAM-60  and 
HATS  (because  they  must  be  removed  from  CVs  to  make  room  for  CASS),  and 
emerging  systems  such  as  the  F-14D  and  A- 12.  [Ref.  50] 

•  Creating  a  centralized  test  and  integration  facility  (CTIF)  located  at  one  of  the 
NADEPS,  which  TPS  developers  would  use  for  the  integration  phase  of 
development  instead  of  using  government  furnished  CASS  stations  in  their  own 
contractor  facilities  as  originally  planned.  This  is  expected  to  produce  a  20% 
savings  in  station  quantity  requirements,  reducing  TPS  development  stations  from 
51  to  41.  [Ref.  53] 

The  planned  operational  testing  of  CASS  and  all  CASS  TPSs  before  fleet 
introduction  is  a  critical  step  in  minimizing  risk  in  many  aspects  of  the  CASS  introduction 
plan.  TPSs  have  never  before  had  thorough  testing  prior  to  their  fleet  introduction.  [Ref. 
25]  As  a  result,  fleet  users  have  experienced  many  problems  with  the  TPS  for  existing 
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ATE  systems,  such  as  problems  with  equipment  supportability  and  software  accuracy,  and 
workload  analysis  has  been  conducted  using  inaccurate  EMT  and  A0  values. 

Workload  analysis  is,  in  the  end,  the  deciding  factor  on  whether  or  not  the 
CASS  program  achieves  its  goal  of  100%  improvement  in  throughput.  Decision  makers 
may  opt  to  abandon  this  goal,  however,  if  the  reduced  cost  of  weapon  system  spares 
resulting  from  doubling  throughput  is  less  than  the  procurement  and  support  costs  of  the 
CASS  stations  needed  to  achieve  it.  Such  economic  tradeoff  analysis  is  beyond  the  scope 
of  current  detailed  planning  but  should  be  a  critical  element  of  future  CASS  introduction 
planning. 

It  is  important  to  note  that  CASS  will  not  significantly  improve  weapon  system 
readiness  or  availability  even  if  it  does  improve  throughput  by  100%.  In  the  long  ran, 
when  ASO  rules  are  used  to  compute  spares  allowances,  all  improvements  in  CASS  TAT 
and  supportability  will  be  reflected  in  reduced  requirements  for  weapon  system  spares. 
This  reduction  in  spares  will  cancel  out  most  improvements  in  customer  support  that 
result  from  the  introduction  of  CASS.  Because  of  its  flexibility  in  spreading  workload 
over  many  redundant  CASS  stations,  CASS  will,  however,  improve  customer  support 
when  there  are  short  ran  surges  in  UUT  inductions  at  a  site.  Present  ATE  systems,  with 
only  a  few  stations  of  each  type  and  no  transportability  of  TPS,  only  build  backlog. 

C,  PROPOSED  METHODS  FOR  CALCULATING  CASS  AVAILABILITY 

The  previous  section  showed  the  importance  of  the  measures  of  CASS  availability 
for  workload  analysis  and  for  determining  CASS  SRA  spares  allowances  using  a  readiness 
based  sparing  model.  Both  of  these  requirements  for  measures  of  availability  call  for  a 
site-wide  computation  of  average  availability.  However,  there  are  two  significant 
problems  with  using  the  standard  A„  formula  for  ATE  in  general  and  for  CASS  in 
particular.  These  problems  are,  first,  the  standard  A,,  formula  fails  to  account  for 
’partially  up’  stations  that  can  test  some  assigned  UUTs  but  not  others.  If  one  assumes 
that  any  failure  downs  the  station,  the  actual  probability  that  CASS  can  perform  its 
mission  when  called  upon  is  underestimated.  Second,  the  problem  of  measuring  CASS 


availability  is  complicated  by  the  commonality  of  CASS  circuit  cards  across 
configurations  and  the  transportability  of  CASS  TPSs. 

For  example,  assume  that  a  CASS  EO  station  has  a  fault  in  a  ’core’  circuit  card  that 
is  not  needed  to  test  the  EO  UUTs  (those  that  are  mapped  only  to  the  EO  configuration) 
but  causes  the  station  to  lose  its  capability  to  test  some  UUTs  normally  run  by  the  hybrid 
station.  Should  this  degradation  impact  the  EO  station’s  measure  of  availability  until  it 
is  repaired?  If  so,  by  how  much?  At  what  point  is  the  station  ’down’:  when  the  first 
UUT  testing  capability  is  lost,  or  only  when  there  is  no  redundant  capability  at  the  site 
to  test  that  UUT?  Some  UUTs  will  have  many  redundant  CASS  stations  able  to  support 
them  (so  loss  of  capability  in  one  station  may  not  be  a  significant  problem)  while  others 
will  have  few  or  none.  Redundant  capability  among  CASS  stations  causes  many  complex 
interactive  effects  on  site-wide  CASS  availability  that  should  be  considered  in  workload 
analysis  and  sparing,  but  are  not  accounted  for  in  a  simple  average  A„  computation. 

AIMD  production  managers  and  aircraft  operational  decision  makers  also  need  a 
more  meaningful  ’snapshot’  measure  of  CASS  availability  than  the  standard  A^  formula 
or  than  a  simple  labeling  of  stations  as  ’up’  or  ’down,’  especially  in  situations  that 
demand  a  realistic  understanding  of  a  ship  or  shore  AIMD  weapon  system  support 
posture.  Managers  will  be  concerned  not  only  with  the  degree  of  redundancy  if  a  station 
loses  the  capability  to  test  a  particular  UUT,  but  with  the  expected  induction  rate  of  the 
UUT.  From  a  readiness  perspective,  degradation  or  loss  of  capability  to  test  a  UUT  with 
a  high  failure  rate  is  of  much  greater  concern  than  a  UUT  that  is  seldom  inducted  by  the 
AIMD. 

To  better  satisfy  these  diverse  requirements  for  measuring  CASS  availability  (for 
planning  and  for  control  purposes)  the  remainder  of  this  section  explains  a  proposal  for 
alternate  methods  of  computing  CASS  availability.  The  first  part  considers  the  problem 
of  establishing  adequate  control  measures  of  availability  and  the  second  part  considers  the 


problem  of  finding  a  planning  measure  useful  for  workload  analysis  and  sparing.  The 
CASS  OMS  system  should  be  capable  of  providing  input  data  for  the  calculations.  These 
proposals  are  based  on  interviews  with  and  notes  from  Professor  T.  P.  Moore. 
[Ref.  54] 

1.  Measures  of  Effectiveness  for  Control  Purposes 

Two  MOEs  for  management  control  are  proposed.  The  first  is  a  point 
availability  measure  labeled  A^.  Its  purpose  is  to  provide  a  snapshot  of  the  overall 
availability  of  the  CASS  station  repair  capacity  (weighted  by  demand  repair  rates)  at  the 
shore  or  carrier  AIMD.  The  purpose  of  the  second,  labeled  AM,  is  to  provide  a  snapshot 
of  the  worst  (minimum)  UUT  repair  capability  at  the  site  to  indentify  where  the  most 
degraded  capability  lies. 

a.  Point  Availability  (AJ 

Ap  is  computed  by  the  following  equation: 

e  hh 

A  »=1  H* 

=  - » 

p  m 

E  \t 

i= 1 

where: 

X,  =  Failure  rate  of  the  i'h  UUT  for  the  given  deckload  and 
flying  hour  program; 

x,  =  Number  of  CASS  stations  able  to  repair  the  irtl  UUT 
at  the  time  of  the  snapshot; 

n,  =■  Number  of  CASS  stations  able  to  repair  the  ith  UUT  if  all  functions  are 
’up’  on  all  stations  that  are  supposed  to  have  repair  capability  for  this 
UUT  (determined  by  the  SSM  mapping  model); 
m  -  Number  of  types  of  UUTs. 
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The  value  Xj/n,  represents,  for  the  i01  UUT,  the  fraction  of  the  AIMD’s 
intended  repair  capability  that  is  available.  The  X,’s  can  be  viewed  as  weights  in  the  A,, 
computation  (i.e.,  A,,  is  a  weighted  average  of  the  values).  The  sum  of  the  A,’s  in 
the  denominator  is  the  normalization  factor  which  makes  the  weights  sum  to  1.  The 
weights  give  greater  emphasis  to  the  status  of  CASS  repair  capability  for  UUTs  that  fail 
more  often. 

Thus  Ap  is  a  function  of  the  status  of  the  CASS  station  functional 
capabilities,  the  aircraft  deckload  and  the  operating  tempo. 

b.  Minimum  Availability  (Au) 

There  are  two  approaches  to  this  ’snapshot’  measure.  The  first  is  to 
compute  it  as  follows: 


,  X,  X ,  X, 

Au  =  min  (  — ,  — ,  — , 

n,  n2  n3 


n 


m 


This  simply  identifies  the  U’JT  type  with  the  worst  case  status  in  terms 
of  the  fraction  of  its  intended  CASS  repair  capability  that  is  actually  available. 

The  second  approach  is  to  combine  the  first  idea  with  the  ’importance’ 
of  the  need  to  repair  a  UUT  provided  by  the  A,’s.  to  this  case,  AM  is  computed  as 
follows: 


r  Xi 

Am  =  max  (  X,  (1 - )  |  for  all  i  -  1,  ...,  m  1. 

ni 

Note  that  (1  -  x/n,)  is  the  unavailable  fraction  of  the  repair  capacity  for 
UUT,.  If  duplicate  TPSs  for  certain  high  failure  rate  UUTs  are  procured  to  enable 
simultaneous  testing  on  multiple  stations  (one  TPS  per  station  able  to  support  these 
UUTs),  the  fraction  of  failures  of  the  i‘h  UUT  that  could  not  be  repaired  due  to  downed 
station  capability  is  represented  by  X,(l  -  x,/n,). 
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c.  Example  Computations 

Table  Vm  shows  example  X’s  for  three  UUTs  and  the  status  of  four 
CASS  configurations  used  to  test  them.  Shaded  blocks  represent  lack  of  repair  capability 
for  a  particular  UUT  as  determined  by  the  SSM  mapping  model.  For  example,  station 
B  is  intended  to  have  the  capability  to  repair  UUT  2  and  3  but  not  1.  ’Up*  and  ’down’ 
indicate  the  actual  functional  status  of  the  capability  within  a  station  at  the  time  of  the 
snapshot.  For  example,  station  C  has  a  ’down’  status  for  repair  of  UUT  1  caused 
(perhaps)  by  a  failed  circuit  card  that  has  not  yet  been  removed  and  replaced.  Given  this 
information,  A,,  and  Am  are  computed  as  follows: 

Table  VIII.  Example  ’Snapshot’  Data  for  Computing  Ap  and  AM 


UUT 

Sta  A 

Sta  B 

Sta  C 

Sta  D 

n 

1 

UP 

DOWN 

UP 

.002 

2/3  =  .6667 

2 

UP 

DOWN 

.005 

1/2  =  .5000 

3 

DOWN 

UP 

DOWN 

DOWN 

.003 

1/4  =  .2500 

(Shaded  blocks  indicate  no  repair  capability) 


Ap  =  .002  (.6667)  +  .005  (.5000)  +  .003  (.2500)  =  .4583 
.002  +  .005  +  .003 

Am,  using  method  1,  is 

Am  =  min  {.6667,  .5000,  .2500)  =  .2500. 

UUT  3  has  the  minimum  value  of  x/n,  and  is  thus  the  most  critical  one  identified  by  this 
method. 
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Am,  using  method  2,  is 


Am  =  max  {.002(1  -  .6667),  .005(1  -  .5000),  .003(1  -  .2500)} 

=  max  {.6667,  2.500,  2.250)}  =  2.500. 

UUT  2  has  the  maximum  value  and  is  thus  the  most  critical  one  identified  by  this 
method. 

2.  Measures  of  Effectiveness  for  Planning  Purposes 
a.  Average  Availability  (Ang) 

The  proposed  MOE  for  planning  such  as  workload  analysis  and  sparing 
will  be  called  A.vg.  It  is  computed  by  the  following  series  of  equations: 

MTBFU 

yj  _  _ V 

v  MTBFy  +  MDTy 

where 

Ay  =  Operational  availability  of  the  capability  to 
repair  UUT,  on  CASS  station^; 

MTBF,j  =  Mean  time  between  failure  of  the  repair  capability 
for  UUT,  on  CASS  station^ 

MDT,j  =  Mean  downtime  for  the  specific  functional 
capability  to  repair  UUT,  on  CASS  station^ 

Next,  define  a  matrix, 

B  =  [by], 

where 

by  =  Ay  if  CASS  stationj  is  supposed  to  have  capability 
(determined  by  the  mapping  model)  to  repair  UUT,; 
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b,j  =  0  otherwise, 
and  another  matrix  M, 

M  =  [my], 

where 

my  =  1  if  CASS  station^  is  supposed  to  have  capability  to 
repair  UUT,; 
niy  =  0  otherwise. 

The  demand  weighted  average  availability  is  then: 

,  KB  I 
**  AM/ 

where 

A,  =  A  vector  of  A,  values; 

A  B  I  =  The  weighted  sum  of  the  A,Ay’s  for  all  UUTs  and  CASS  stations; 

X  M  I  =  The  sum  of  A,miJ  for  all  UUTs  and  CASS  stations. 

b.  Example  Computations 

Table  IX  shows  the  three  UUTs  and  four  CASS  stations  presented  above 
with  values  of  Ay  for  the  capability  to  repair  UUT,  using  CASS  station;.  Given  the  values 
of  availability  for  each  repair  capability,  the  site-wide  average  availability  (A,vg)  is 
computed.  Note  also  that  average  availability  can  be  computed  for  an  individual  CASS 
station  and  for  an  individual  UUT  repair  capability  (across  the  CASS  stations). 
Site-wide  average  availability  is: 

A,vg  =  r.002  (.14  +  .81  +  .70)1+r.005  (.77  ±  .65)l+r.0Q3  (.61  +  .75  +  .69)1  =  .72. 
(.002  *  3)  +  (.005  *  2)  +  (.003  *  3) 
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Table  IX.  Example  Data  for  Computing  Average  Availabilty  (A,vg) 


UUT 

Sta  A 

Sta  B 

Sta  C 

Sta  D 

X 

1 

.74 

— 

.81 

.70 

.002 

2 

■BbIB 

.77 

.65 

.005 

3 

.67 

.75 

.69 

.003 

(Shaded  blocks  indicate  no  repair  capability) 


UUT  average  availabilities  are: 

Auur  i  =  -74  4-  .81  4-. 70  =  .75; 

3 

Auxjt  2  =  -77  +  .65  =  .71; 

2 

Auxn-j  =  .67  +  .15  +.69  =  .70. 

3 

CASS  station  average  availabilities  are: 

Ac  ass  a  =  (.002  *  .74)  ±  (.003  *  .67)  =  .70; 

.002  +  .003 

Acassb  =  (.005  *  .77)  ±  (.003  *  .75)  =  .76; 

.005  +  .003 

Acassc  =  (-002  *  .81)  +  (.003  *  .69)  =  .74; 

.002  +  .003 

Acass  d  =  (.002  *  .70)  +  (.005  *  .65)  =  .66. 

.002  +  .005 
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D.  EXPANDABILITY 


The  second  criteria  for  evaluating  CASS  in  comparison  to  existing  ATE  systems  is 
its  ability  to  accept  emerging  testing  requirements  without  the  need  for  major  hardware 
and  software  changes.  This  expandability  has  become  a  critical  requirement  for  CASS 
as  a  result  of  quantum  leaps  in  the  technology  used  in  weapon  systems  now  under 
development  which  have  resulted  in  new  frequencies,  power  levels  and  functional 
parameters  that  exceed  current  testing  capabilities.  Advanced  technologies  which  CASS 
must  support  in  the  future  include:  [Ref.  13:p.  5-11] 

•  Digital  technology  using  very  high  speed  integrated  circuits  (VHSIC). 

•  Advanced  electro-optic  systems  such  as  high  energy  C02  lasers  and  focal  plane 
array  technology. 

•  Millimeter  wave  technology  for  electronic  warfare  systems  using  very  high 
frequencies. 

•  Advanced  radar  and  data  transmission  systems  using  spread  spectrum  technology 
to  achieve  high  frequencies  and  bandwiths. 

•  Fiber-optics  interface  systems  that  will  replace  existing  data  transmission  cables. 
Past  ATE  systems  have  not  been  able  to  expand  to  keep  pace  with  changes  in 

technology.  The  result  has  been  obsolescence  cycles  of  seven  to  fourteen  years.  The 
FOT  was  created  to  reduce  the  proliferation  of  testers,  yet  the  inflexibility  of  these 
systems  to  accept  new  technology  has  resulted  in  the  continued  need  for  peculiar  support 
equipment.  The  Navy  ATE  inventory  today  is  a  mixture  of  more  than  300  PSE  testers 
and  24  ATE  systems.  Whenever  a  new  test  requirement  emerges,  weapon  system 
program  managers  look  at  the  existing  inventory  and  try  to  adapt  an  FOT  tester  to  fit  the 
need,  either  through  station  modifications  or  development  of  complex  IDs  that  contain  the 
missing  instruments.  If  neither  of  these  are  feasible,  NAVAIR  buys  another  PSE  tester. 
[Ref.  9] 

A  1987  cost-benefit  analysis  conducted  for  NAVAIR  by  Prospective  Computer 
Analysts  (PCA)  estimated  the  extent  and  cost  of  FOT  modifications  needed  to 
accommodate  projected  requirements  through  the  year  2010.  It  identified  seven  members 
of  the  FOT  capable  of  receiving  new  instrumentation  to  provide  the  needed  capabilities 
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and  eliminate  most  PSE.  These  include  HTS,  CAT  HID,  RADCOM,  RADCOM-Radar 
Interface  Unit  (RIU),  EWTS,  TMV  and  IMUTS  H.  The  existing  electro-optics  tester 
cannot  be  modified  to  accept  the  new  requirements  and  a  new  one  will  be  needed.  The 
conclusions  of  this  study  were  that  development  and  procurement  costs  of  this 
’augmented’  FOT  far  exceeded  the  costs  of  CASS  (to  be  discussed  further  in  the  next 
section)  even  though  the  FOT  systems  chosen  are  mature  and  would  require  only 
replacement  of  instruments.  They  would  also  require  considerably  more  facility  space; 
an  average  carrier  would  require  34  augmented  FOT  (AFOT)  stations  compared  to  19 
CASS  stations.  [Ref.  l:p.  5-5] 

There  are  many  factors  which  limit  the  growth  capability  of  commercial  test 
equipment  (CTE)  based  testers.  Among  the  most  serious  constraints  are: 
[Ref.  55] 

•  Space:  New  instruments  must  fit  physically  into  existing  racks. 

•  Heat:  New  instruments  must  not  exceed  the  existing  air  conditioning  capacity  of 
facilities. 

•  Speed:  New  instmments  may  drive  changes  in  the  ATE  operating  system,  test 
executive  or  application  software. 

•  Calibration:  New  instruments  must  not  exceed  existing  calibration  accuracy. 
CASS  must  overcome  these  problems  to  achieve  its  goal  of  a  20-year  life  cycle  and 

a  halt  in  the  proliferation  of  PSE.  The  CASS  approach  to  these  goals  is  termed  "flexible 
and  open-ended  architecture"  centering  around  the  functions-on-a-circuit-card  described 
in  Chapter  1H.  This  approach  is  intended  to  enable  adding  functions  to  meet  new  testing 
requirements  simply  by  inserting  an  additional  card  or  replacing  an  existing  one.  Because 
CASS  separates  the  hardware  from  the  application  software  with  intermediate 
microprocessor  asset  controllers,  functions  may  change  or  move  around  within  the  tester 
without  impact  on  the  TPS  and  station  computer  software.  Any  change  requires  only  a 
new  address  within  the  asset  controller  software.  Functional  assets  may  also  be  combined 
by  the  asset  controllers  to  create  large  functional  test  suites  of  stimulus  and  response 
capabilities  required  for  complex  testing  strategies.  [Ref.  28 :p.  2] 
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Armed  for  the  first  time  with  a  test  system  capable  of  low  cost  expansion  to  meet 
new  technologies,  NAVAIR  has  established  the  policy  that  CASS  will  always  be  modified 
in  response  to  emerging  requirements  rather  than  acquiring  PSE.  At  least  thirty  months 
before  a  new  testing  capability  is  needed  in  the  fleet,  NAVAIR  (AIR-552)  will  match  the 
attributes  of  CASS  to  the  technical  test  requirements  of  the  new  weapon  system  using  the 
SSM  mapping  model.  If  the  requirement  cannot  be  matched  to  an  existing  CASS 
functional  asset,  one  of  two  actions  will  be  taken.  If  the  test  requirement  falls  within  the 
scope  of  the  CASS  perfoimance  specification,  an  engineering  change  proposal  (ECP)  will 
be  initiated  using  support  equipment  procurement  funds.  If  the  requirement  falls  outside 
the  CASS  scope,  a  change  will  be  initiated  under  the  preplanned  product  improvement 
(P3!)  program  using  R&D  funds. 

As  site-specific  testing  requirements  and  workload  change  over  time  with  the  phase- 
in  and  phase-out  of  weapon  systems,  CASS  stations  will  not  only  be  modified  to  meet 
new  functional  requirements  but  will  be  reconfigured  to  maintain  an  optimal  workload. 
For  example,  if  a  site  has  three  hybrid  testers  and  one  EO  tester  and  additional  EO  testing 
requirements  emerges,  one  of  the  three  hybrid  testers  could  be  reconfigured  into  a  second 
EO  tester  by  adding  the  needed  circuit  cards,  EO  rack  and  ancillary  equipment.  The  new 
EO  tester  would  be  technically  able  to  support  the  new  requirement  plus  the  full  set  of 
UUTs  previously  supported  by  the  hybrid  tester.  [Ref.  28:p.  4]  Through  these 
approaches,  CASS  should  hopefully  be  able  to  meet  the  challenge  of  new  technology  and 
remain  the  Navy’s  standard  general  purpose  tester. 


E.  ATE  SYSTEM  LIFE  CYCLE  COSTS 

The  final  criterion  for  evaluating  CASS  is  the  extent  to  which  it  reduces  ATE 
system  life  cycle  costs.  For  this  criterion  CASS  must  be  compared,  not  to  existing 
systems,  but  to  an  Augmented  Family  of  Testers  which  is  the  only  feasible  alternative  to 
CASS  for  the  expected  twenty  year  life  of  the  program. 

Table  X  presents  the  conclusions  of  the  1987  PCA  cost-benefit  analysis.  [Ref.  l:p. 
1-3]  They  determined  that  the  20  year  costs  of  procuring  an  augmented  FOT  of  eight 
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testers,  matched  instrument-by-instrument  to  current  CASS  testing  capabilities,  would 
exceed  the  costs  of  CASS  by  $2,022  million. 

Table  X.  1987  Cost-Benefit  Analysis  of  Procurement  of  CASS  versus  AFOT 


Cost  Catagory 

Augmented  FOT 

CASS 

ATE  Development 

227 

164 

ATE  Production 

3,777 

1,178 

TPS  Procurement 

8,291 

9,202 

Storage/Transport 

367 

96 

Total 

12,662 

10,640 

(Costs  are  in  $  Millions) 


This  study  ignored  support  costs,  and  considered  only  the  costs  of  development  and 
production  of  ATE  stations  and  TPS  plus  storage  and  transportation  during  phase-in.  It 
also  computed  relative  cost,  schedule  and  performance  risks  for  each  alternative,  which 
it  found  to  be  roughly  equivalent.  Its  most  significant  assumptions  were: 

•  ATE  station  costs:  CASS  development  and  production  costs  are  based  on  the  FSED 
contract  price.  AFOT  development  and  production  costs  are  based  on  vendor 
catalogue  costs  for  CTE  and  complexity  factors  and  standard  engineering  labor  hour 
costs  for  CDE. 

•  TPS  procurement:  Costs  per  UUT  are  equivalent  for  CASS  and  AFOT.  Total 
CASS  TPS  costs  exceed  those  of  AFOT  by  $911  million  due  solely  to  the  costs  of 
transitioning  existing  testers  to  CASS. 

•  Workload  analysis:  The  number  of  CASS  stations  required  was  based  on  the  rough 
estimates  computed  by  NAVA1R  early  in  the  full  scale  development  phase.  The 
number  of  AFOT  stations  was  based  on  the  current  throughput  and  UUT  load  of 
the  seven  testers  that  would  be  augmented,  assuming  a  proportional  increase  in 
stations  required  to  support  the  added  workload  of  UUTs  from  off-loaded  testers 
and  new  weapon  systems. 

•  Costs  are  computed  in  1987  dollars  with  no  discounting  of  future  dollars  to  present 
values. 
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The  overall  effect  of  these  assumptions  is  an  underestimation  of  the  cost  advantages 
of  CASS.  TPS  development  and  production  costs  for  CASS  will  almost  certainly  be  less 
than  for  development  of  the  same  TPS  for  augmented  versions  of  existing  testers  because 
CASS  will  require  fewer  complex  and  costly  IDs  than  would  the  less  flexible  augmented 
FOT.  CASS  TPS  development  tools  (discussed  in  Chapter  HI)  also  offer  cost  advantages. 
NAVAIR  has  not  quantified  the  resulting  reduction  in  TPS  development  costs  but 
considers  it  to  be  significant.  The  study  also  ignored  the  significant  advantages  of  CASS 
compared  to  the  FOT  in  such  areas  as  TAT,  supportability,  flexibility,  TPS 
transportability,  and  expandability,  and  assumed  that  a  tester  matched  functionally  to 
CASS  is  equivalent  in  performance.  This  is  not  a  valid  assumption,  and  results  in  further 
underestimation  of  the  benefit/cost  ratio  of  CASS.  [Ref.  56] 

NAVAIR  also  conducted  a  cost  benefit  analysis  of  CASS  that  includes  many  of  the 
operation  and  support  costs  neglected  in  the  PCA  study.  Table  XI  compares  the  February 
1990  NAVAIR  estimates  of  the  20  year  life  cycle  costs  of  CASS  versus  a  "future"  FOT 
and  PSE  combination.  [Ref.  57]  This  analysis  estimated  CASS  life  cycle  costs 
to  be  $8,069  million  less  than  the  alternative. 

Table  XI.  1990  Life  Cycle  Cost  Comparison  for  CASS  versus  a  Future  FOT 


Cost  Catagory 

Future  FOT 

CASS 

ATE  Development 

653 

180 

ATE  Production 

8,452 

1,312 

Operation/Support 

2,066 

1,440 

P3! 

- 

170 

Total 

11,171 

3,102 

(Costs  are  in  $  Millions) 
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These  estimates  are  in  undiscounted  1989  dollars,  and  include  all  operation  and 
support  costs  except  manpower.  The  expected  costs  of  CASS  P*I  are  also  included.  They 
do  not,  however,  include  the  costs  of  TPS. 

The  1987  PCA  study  estimated  CASS  development  and  production  costs  to  be 
$2,662  million  cheaper  than  AFOT,  while  the  1990  NAVAIR  study  estimated  these  CASS 
costs  to  be  $7,613  million  cheaper  than  a  future  FOT.  The  differences  in  their 
conclusions  are  due  primarily  to  different  assumptions  regarding  hardware  requirements 
for  the  non-CASS  alternative.  Both  analyses,  however,  agree  that  CASS  is  the  cheaper 
alternative  by  a  significant  margin. 

Because  CASS  is  also  cheaper  in  TPS  and  operation  and  support  costs,  it  is  clearly 
the  least  cost  alternative  for  satisfying  NAVAIR  automatic  testing  needs  throughout  its 
life  cycle. 
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VI.  SUMMARY,  CONCLUSIONS  AND  RECOMMENDATIONS 


A.  SUMMARY 

CASS  will  enter  operational  service  in  fleet  AlMDs  beginning  in  mid- 1993.  This 
thesis  attempted  to  answer  the  questions  most  in  the  minds  of  fleet  users  who  will  have 
to  live  with  CASS,  for  better  or  worse.  Will  it  solve  the  host  of  ATE  problems  that 
currently  plague  the  fleet?  What  is  NAVAIR’s  strategy  for  solving  them,  and  how  does 
it  differ  from  past  strategies?  Will  CASS  make  a  difference  in  terms  of  weapon  system 
support? 

To  lay  the  groundwork  for  answering  the  research  questions,  the  thesis  introduced 
ATE  by  describing  its  place  in  the  Naval  avionics  maintenance  concept,  ATE  maintenance 
facilites  and  architecture.  It  then  summarized  ATE  evolution  to  show  the  problems  and 
initiatives  that  led  up  to  CASS.  It  next  described  the  CASS  program  goals,  system, 
acquisition  strategy  and  introduction  plan.  Because  achieving  these  goals  and  solving 
current  AT  E  problems  are  rooted  in  the  avionics  repair  process,  the  thesis  described  this 
process  and  its  relationship  to  weapon  system  support  and  spares  allowances.  Finally,  the 
thesis  evaluated  CASS  in  comparison  to  the  existing  family  of  testers  using  three  criteria: 
throughput,  expandability  and  test  equipment  life  cycle  cost. 


B.  CONCLUSIONS 

The  primary  conclusion  of  this  analysis  is  that  CASS  is  technically  capable  of 
achieving  its  program  objectives  and  solving  or  alleviating  virtually  every  significant 
problem  faced  by  AlMDs  in  operating  and  supporting  the  current  family  of  ATE.  Its 
objectives  are  first,  to  improve  throughput  100%  in  order  to  improve  weapon  system 
operational  availability,  readiness  and  capability  to  meet  combat  sortie  requirements;  and 
second,  to  standardize  ATE  hardware,  software  and  support  in  order  to  reduce  the  cost 
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of  ownership  of  both  avionic  systems  and  support  equipment.  These  objectives  address 
the  most  significant  ATE  problems  facing  the  fleet  today:  those  related  to  ATE 
supportability  and  its  affects  on  weapon  system  readiness  and  support  costs. 

The  CASS  acquisition  strategy,  support  philosophy  and  equipment  design  are 
significant  improvements  over  those  of  most  existing  ATE  systems  in  terms  of 
supportability  and  life  cycle  costs.  CASS  is  a  model  development  program  in  many 
respects.  However,  there  are  still  many  areas  of  risk  remaining  in  the  program.  CASS’ 
success  in  the  fleet  is  critically  dependent  on  decisions  that  will  be  made  during  TPS 
development  and  introduction  planning.  The  areas  of  risk  with  the  most  damaging 
consequences  are: 

•  Funding  shortfalls  for  CASS  station  procurement  or  TPS  development  which  may 
disrupt  the  sequence  of  introductions  and  transitions,  result  in  less  than  optimal 
profiles  of  quantities  versus  configurations  of  CASS  stations  delivered  to  individual 
sites,  result  in  less  than  optimal  TPS  fault  isolation  or  maintainability  performance, 
and  lead  ultimately  to  greater  weapon  system  life  cycle  support  costs.  Slippages 
in  the  introduction  and  transition  schedule  are  likely  to  result  in  increased  interim 
support  costs  and  cause  a  ripple  effect  in  other  programs.  A  one  year  slip  in  F-14D 
TPS  development,  for  example,  would  increase  F-14D  support  costs  by 
approximately  $30  million  and  disrupt  the  carrier  transition  schedule. 

•  Inaccurate  workload  analysis  of  site-specific  station  requirements.  If  this  analysis 
results  in  delivery  of  too  few  CASS  stations  it  may,  like  funding  shortfalls,  lead  to 
weapon  system  supportability  problems  and  higher  than  optimal  weapon  system 
support  costs. 

The  following  paragraphs  summarize  the  conclusions  of  the  study  in  greater  detail 
with  regard  to  the  three  evaluation  criteria. 

1.  Throughput 

As  a  result  of  a  number  of  attributes,  CASS  is  likely  to  substantially  improve 
throughput  in  comparison  to  existing  testers.  CASS  attributes  which  favorably  influence 
throughput  include  use  of  a  functional  test  strategy,  expanded  computer  capabilities,  and 
improved  supportability.  Use  of  a  functional  test  strategy  is  likely  to  reduce  TAT.  BCM 
rate  and  cost  of  weapon  system  spares  for  at  least  some  UUTs,  especially  for  newer 
weapon  systems  having  extensive  BIT  capability. 


Advances  in  computer  capabilities  between  the  family  of  testers  (FOT)  and 
CASS  may  significantly  reduce  the  relative  time  required  for  several  testing  and 
management  functions.  These  capabilities  include  expanded  computer  processing  power, 
which  allows  both  faster  testing  and  simultaneous  performance  of  multiple  tasks,  and 
management  tools  such  as  the  Operational  Management  System  (OMS)  and  Intermediate 
Maintenance  Operations  Management  System  (IMOMS).  Coupled  with  the 
transportability  of  many  weapon  system  TPS  between  different  CASS  configurations, 
these  management  tools  offer  the  capability  to  efficiently  and  flexibly  manage  a  network 
of  CASS  stations.  This  flexible  network  is  a  key  to  minimizing  backlog  and  improving 
weapon  system  support  because  it  enables  the  CASS  testing  load  to  be  evenly  distributed 
across  many  avionics  shops.  Existing  ATE  systems  have  little  such  flexibility  and 
redundant  testing  capability.  As  a  result,  they  commonly  build  up  backlog  during  periods 
of  increased  operating  tempo  or  even  due  to  random  variations  in  induction  rates. 

Improvements  in  CASS  supportability  will  work  to  increase  throughput  in  two 
primary  ways.  First,  better  self-testability  and  maintainability  will  reduce  elapsed 
maintenance  time  (EMT)  because  less  time  will  be  lost  troubleshooting  station  and  TPS 
faults  during  on-line  UUT  testing  and  repair.  TPS  maintainability  is  UUT  specific, 
however,  so  it  is  still  an  unknown  element  (although  NAVAIR  has  established  tentative 
guidelines  for  TPS  development  which  would  lead  to  satisfactory  levels  of 
maintainability).  It  is  a  function  primarily  of  the  comprehensiveness  of  TPS  self-testing 
capability  (which  improves  maintainability)  and  the  amount  of  active  circuitry  they 
contain  (which  reduces  it).  Because  TPS  maintainability  affects  throughput  and  weapon 
system  spares  costs,  TPS  development  should  seek  an  optimum  (least  total  weapon  system 
support  cost)  tradeoff  between  total  TPS  costs  and  total  spares  costs.  This  tradeoff  must 
also  be  considered  in  any  decision  to  develop  active  interconnection  devices  rather  than 
modify  the  CASS  stations  to  attain  the  capability  to  support  new  technology. 

The  second  way  that  CASS  supportability  can  increase  throughput  is  by 
improving  CASS  station  operational  availability.  The  analysis  did  not  provide  an 
unequivocal  answer  on  whether  CASS  will  have  a  higher  level  of  availability,  however. 
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It  is  likely  that  CASS  stations  will  be  far  more  maintainable  (measured  by  mean  time  to 
repair)  than  many  existing  testers  such  as  RADCOM  and  CAT  HID  because  CASS 
minimizes  the  use  of  commercial  test  equipment  and  provides  extensive  self-testability, 
modularity  and  improved  operator  support  aids  such  as  an  automatic  technical  information 
system  (ATI)  and  embedded  training.  CASS  may  be  less  reliable  than  these  same  testers, 
at  least  upon  initial  fleet  introduction,  if  its  comparatively  low  mean  time  between  failure 
assumptions  are  accurate.  Its  reliability  is  likely  to  grow  during  operational  service, 
however,  and  should  eventually  match  or  exceed  the  reliability  of  existing  testers.  The 
CASS  requirement  for  a  portable  chiller  for  station  cooling  may  negatively  impact  its 
reliability. 

The  most  significant  factor  affecting  ATE  operational  availability  is  mean 
logistics  delay  time  (MLDT).  The  conclusion  of  this  analysis  is  that  CASS  will 
substantially  improve  MLDT  in  comparison  to  existing  testers  because  it  will  have  an 
organic  repair  capability  and  all  logistic  support  will  be  delivered  up  front  and  tested 
during  OPEVAL  rather  than  phased  in.  Because  of  this,  CASS  operational  availability 
will  likely  be  higher  than  that  of  existing  testers. 

Given  a  particular  value  for  CASS  operational  availability  and  the  EMT  of 
each  UUT,  workload  analysis  facilitates  additional  tradeoff  decisions  to  achieve  the 
optimal  total  weapon  system  support  cost.  Attaining  the  CASS  program  goal  of  a  100% 
increase  in  throughput  may  not  be  desirable  because  throughput  is  intimately  related  to 
many  factors,  including  the  number  and  cost  of  stations  at  a  site,  the  cost  of  CASS  SRA 
sparing,  the  cost  of  weapon  system  sparing  and  the  cost  of  TPS  procurement.  The  CASS 
System  Synthesis  Model  (SSM)  is  under  development  to  enable  consideration  of  at  least 
some  of  these  many  complex  relationships  in  calculating  optimal  combinations  of  CASS 
stations  at  each  site.  Its  use  is  hampered  at  the  present  stage  of  introduction  planning, 
however,  by  the  lack  of  solid  data  on  TPS  EMT,  difficulty  in  calculating  a  value  for 
CASS  availability  that  accounts  for  partially  capable  stations  and  the  interactive  effects 
of  redundant  testers  at  a  site,  and  the  SSM’s  lack  of  a  queuing  simulation  model  for 
workload  analysis.  The  linear  programming  model  currently  used  in  the  place  of  such  a 
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simulation  model  is  far  less  accurate  and  could  lead  to  substantial  errors  in  site  planning, 
as  has  occurred  in  past  ATE  programs  using  such  a  model,  and  could  result  in  many  of 
the  negative  consequences  highlighted  at  the  start  of  this  section.  Operational  testing  of 
CASS  and  CASS  TPSs  will  substantially  reduce  the  risk  in  measuring  these  input 
parameters  and  is  critical  to  the  program’s  success. 

2.  Expandability 

Expandability  to  accept  emerging  testing  requirements  is  a  critical  requirement 
for  CASS  to  be  able  meet  the  challenge  of  new  technology  and  remain  the  Navy’s 
standard  tester  throughout  its  intended  20  year  life.  The  conclusion  of  this  thesis  is  that 
CASS  is  clearly  far  more  able  than  existing  testers  to  keep  pace  with  changes  in 
technology.  It  achieves  this  expandability  because  it  separates  the  hardware  from  the 
software  and  because  of  its  flexible  ’function-on-a-card’  approach  that  enables  functions 
to  be  added  by  inserting  new  cards  or  replacing  existing  cards.  Rather  than  procuring  a 
new  tester,  as  has  often  been  done  in  the  past  when  a  new  testing  requirement  emerged, 
NAVAIR  can  initiate  a  change  to  the  CASS  station  using  the  engineering  change  proposal 
(ECP)  or  preplanned  product  improvement  (P3!)  processes.  CASS  stations  can  also  be 
reconfigured  from  one  functional  type  of  station  into  another  as  site-specific  testing 
requirements  and  workload  change  over  time.  These  capabilities  of  functional  expansion 
and  reconfigurability  should  enable  each  AIMD  site  to  maintain  an  optimal  set  of  testers 
to  minimize  the  total  weapon  system  life  cycle  cost. 

3.  ATE  Life  Cycle  Cost 

Two  studies  were  reviewed  which  showed  that  CASS  has  substantially  lower 
20-year  life  cycle  costs  than  an  upgraded  FOT.  A  1987  cost  benefit  analysis  performed 
by  Perspective  Computer  Analysts  (PCA)  compared  CASS  to  an  ’augmented’  family  of 
testers  (AFOT)  composed  of  seven  existing  testers  plus  a  new  electro-optical  (EO)  tester. 
These  eight  festers  would  take  over  virtually  all  avionics  testing  from  the  other  common 
ATE  and  peculiar  support  equipment  (PSE)  and  meet  new  technology  requirements 
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envisioned  through  the  year  2011.  The  study  found  the  cost  of  developing  and  producing 
such  an  AFOT  would  exceed  the  costs  of  CASS  by  $2,022  million.  The  study  included 
the  costs  of  TPS  procurement,  and  found  that  CASS  TPS  would  exceed  the  cost  of  the 
AFOT  TPS  by  $91 1  million  as  a  result  of  the  requirement  to  transition  many  existing  TPS 
to  CASS  support.  The  study  assumed  individual  new  CASS  TPS  development  costs 
would  be  similar  to  typical  costs  for  current  TPS,  which  is  probably  an  invalid  assumption 
since  CASS  is  intended  to  have  less  complex  interconnection  devices.  As  a  result,  new 
CASS  TPS  are  likely  to  be  substantially  cheaper  than  would  TPS  for  an  AFOT  (rather 
than  more  expensive  as  the  PCA  study  concluded).  The  study  did  not  consider  operation 
and  support  costs. 

The  second  cost  benefit  analysis,  performed  by  NAVAIR,  included  operation 
and  support  costs  (except  manpower)  but  not  TPS  costs.  It  found  CASS  development  and 
production  costs  to  be  $7,613  million  cheaper  than  a  ’future*  FOT,  and  total  20-year  life 
cycle  costs  to  be  $8,069  million  cheaper  than  the  alternative.  Although  the  conclusions 
of  the  two  studies  differ  in  many  cost  categories  as  a  result  of  differing  assumptions,  both 
agree  that  CASS  is  the  cheaper  alternative  by  a  significant  margin. 


C.  RECOMMENDATIONS 

The  following  recommendations  are  made  based  on  the  above  conclusions: 

1.  NAVAIR  should  continue  to  fund  the  CASS  program  at  100%  of  requirements  to 
maintain  the  emerging  introduction  and  transition  schedule  and  the  optimal 
(minimum  weapon  system  support  cost)  level  of  TPS  performance.  The 
introduction  plan  should  be  developed  and  justified  through  rigorous  economic 
cost  analysis.  Efforts  to  reduce  current  outlays  by  slipping  the  schedule  or 
reducing  TPS  performance  will  most  likely  lead  to  far  greater  expenditures  in  the 
out  years. 

2.  Expedite  completion  of  development  of  a  queuing  simulation  and  integrate  it  into 
the  System  Synthesis  Model  (SSM)  for  determination  of  CASS  station 
quantities/configurations  to  be  delivered  to  each  site.  Expand  the  model  (perhaps 
by  modifying  the  existing  linear  programming  portion  of  the  workload  model)  to 
consider  the  full  range  of  inter-related  factors  that  should  be  traded  off  to  optimize 
weapon  system  support  costs  at  each  site.  Additional  factors  not  presently 
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included  in  the  SSM  include  the  costs  of  procuring  and  holding  weapon  system 
spares,  the  costs  of  procuring  multiple  TPS  for  high  failure  rate  UUTs,  the  costs 
of  CASS  spares  and  their  effects  on  CASS  availability,  measures  of  CASS 
availability  that  account  for  partial  capability  and  the  interactive  effects  of 
redundant  stations,  variations  in  UUT  inductions  and  the  effects  of  multiple  server 
queuing  on  EMT  and  TAT. 


D.  FOLLOW-ON  RESEARCH 

The  thesis  proposed  new  methods  for  calculating  CASS  availability  for  management 
control  and  introduction  planning.  Additional  research  is  needed,  ideally  by  the  Naval 
Air  Engineering  Center  since  they  maintain  the  SSM,  to  validate  the  usefulness  of  these 
measures  for  solving  some  of  the  current  problems  involved  in  using  measures  of  CASS 
operational  availability  (AJ  for  sparing  and  workload  analysis.  If  these  measures  are 
found  to  be  useful,  research  is  also  needed  to  validate  the  ability  of  the  IMOMS/OMS 
systems  to  collect  the  necessary  data.  If  their  ability  is  found  to  be  inadequate,  these 
systems  should  be  modified. 
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APPENDIX  A:  CASS  INTRODUCTION  AND  TRANSITION  SCHEDULE 

FOR  FISCAL  YEARS  1990-1999 


This  appendix  presents  the  current  schedule  for  introduction  of  new  weapons 
systems  to  CASS  support  and  transition  of  existing  ATE  systems  to  CASS.  It  is  taken 
from  a  May  1990  unpublished  draft  version  of  the  CASS  Introduction  Plan  prepared  by 
the  Naval  Air  Engineering  Center  for  NAVAIR.  [Ref.  57]  Weapon  and  test  systems  are 
listed  in  alphabetical  order.  The  shaded  areas  represent  periods  of  interim  support 
required  prior  to  availability  of  CASS  stations  and  TPS  to  support  a  particular  weapon 
system.  Split  bars  indicate  a  period  of  dual  CASS  and  interim  support.  Various  notations 
indicate  the  type  of  weapon  system  interim  support  that  is  planned,  such  as  contractor 
warrantees,  commercial  factory  test  equipment  (FTE)  or  ’repair  of  repairables’  (ROR), 
organic  depot  support,  existing  PSE  testers  or  existing  ATE  systems  such  as  the 
RADCOM,  CAT  HID  or  ASM-614. 
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ASS  INTRODUCTION  SCHEDULE  SUMMARY  BY  FISCAL  YEAR 


Figure  13.  CASS  Introduction  and  Transition  Schedule,  FY-90  to  FY-99,  Plate  A 


105 


SHADE-AREA  REPRESENTS  RQMT  FOR  INTERIM  SUPPORT 


CASS  INTRODUCTION  SCHEDULE  SUMMARY  BY  FISCAL  YEAR 


Figure  13.  CASS  Introduction  and  Transition  Schedule.  FY-90  to  FY-99,  Plate  B 


106 


SHADE  AREA  REPRESENTS  RQMT  FOR  INTERIM  SUPPORT 


ASS  INTRODUCTION  SCHEDULE  SUMMARY  BY  FISCAL  YEAR 


Figure  13.  CASS  Introduction  and  Transition  Schedule,  FY-90  to  FY-99,  Plate  C 


107 


SHADE  AREA  REPRESENTS  RQMT  FOR  INTERIM  SUPPORT 


ASS  INTRODUCTION  SCHEDULE  SUMMARY  BY  FISCAL  YEAR 


Tf 


Figure  13.  CASS  Introduction  and  Transition  Schedule,  FY-90  to  FY-99,  Plate  D 
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SHADE'AREA  REPRESENTS  RQMT  FOR  INTERIM  SUPPORT 


CASS  INTRODUCTION  SCHEDULE  SUMMARY  BY  FISCAL  YEAR 


Figure  13.  CASS  Introduction  and  Transition  Schedule,  FY-90  to  FY-99,  Plate  E 
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SHADE  AREA  REPRESENTS  RQMT  FOR  INTERIM  SUPPORT 


CASS  INTRODUCTION  SCHEDULE  SUMMARY  BY  FISCAL  YEAR 


SHADE* AREA  REPRESENTS  RQMT  FOR  INTERIM  SUPPORT 
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